Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-30823

%PYTHONPATH% not set in python/docs/make2.bat, resulting in failed/wrong documentation builds

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.4.5
    • 2.4.6
    • Documentation, PySpark, Windows
    • None
    • Tested on Windows 10.

    Description

      When building the PySpark documentation on Windows, by changing directory to python/docs and running make.bat (which runs make2.bat), the majority of the documentation may not be built if pyspark is not in the default %PYTHONPATH%. Sphinx then reports that pyspark (and possibly dependencies) cannot be imported.

      If pyspark is in the default %PYTHONPATH%, I suppose it is that version of pyspark – as opposed to the version found above the python/docs directory – that is considered when building the documentation, which may result in documentation that does not correspond to the development version one is trying to build.

      python/docs/Makefile avoids this issue by setting
      export PYTHONPATH=$(realpath ..):$(realpath ../lib/py4j-0.10.8.1-src.zip)
      on line 10, but make2.bat does no such thing. The fix consist of adding
      set PYTHONPATH=..;..\lib\py4j-0.10.8.1-src.zip
      to make2.bat.

      See GitHub PR #27569.

      Attachments

        Issue Links

          Activity

            People

              DavidToneian David Toneian
              DavidToneian David Toneian
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: