Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-44928

Replace the module alias 'sf' instead of 'F' in pyspark.sql import functions

    XMLWordPrintableJSON

Details

    • Documentation
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.5.0
    • 3.5.0
    • PySpark
    • None

    Description

      from pyspark.sql import functions as F
      

      isn’t very Pythonic - it does not follow PEP 8, see Package and Module Names (https://peps.python.org/pep-0008/#package-and-module-names).

      Modules should have short, all-lowercase names. Underscores can be used in the module name if it improves
      readability. Python packages should also have short, all-lowercase names, although the use of underscores
      is discouraged.

      Therefore, the module’s alias should follow this. In practice, the uppercase is only used at the module/package
      level constants in my experience, see also Constants (https://peps.python.org/pep-0008/#constants).

      See also this stackoverflow comment (https://stackoverflow.com/questions/70458086/how-to-correctly-import-pyspark-sql-functions#comment129714058_70458115).

      Attachments

        Activity

          People

            gurwls223 Hyukjin Kwon
            gurwls223 Hyukjin Kwon
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: