Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-31895

df.explain("mode") should work in PySpark side as well for consistency

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0
    • 3.0.0
    • PySpark
    • None

    Description

      Scala:

      scala> spark.range(10).explain("cost")
      == Optimized Logical Plan ==
      Range (0, 10, step=1, splits=Some(12)), Statistics(sizeInBytes=80.0 B)
      
      == Physical Plan ==
      *(1) Range (0, 10, step=1, splits=12)
      

      PySpark:

      >>> spark.range(10).explain("cost")
      Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
        File "/.../spark/python/pyspark/sql/dataframe.py", line 333, in explain
          raise TypeError(err_msg)
      TypeError: extended (optional) should be provided as bool, got <class 'str'>
      

      Attachments

        Issue Links

          Activity

            People

              gurwls223 Hyukjin Kwon
              gurwls223 Hyukjin Kwon
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: