Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-33392 Align DSv2 commands to DSv1 implementation
  3. SPARK-33889

NPE from SHOW PARTITIONS for null partition values

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.2.0
    • 3.1.0
    • SQL
    • None

    Description

      The following test fails with NPE on V2 table:

        test("null and empty string as partition values") {
          import testImplicits._
          withNamespaceAndTable("ns", "tbl") { t =>
            val df = Seq((0, ""), (1, null)).toDF("a", "part")
            df.write
              .partitionBy("part")
              .format("parquet")
              .mode(SaveMode.Overwrite)
              .saveAsTable(t)
      
            runShowPartitionsSql(s"SHOW PARTITIONS $t", Row("part=null") :: Nil)
          }
        }
      
      java.lang.NullPointerException was thrown.
      java.lang.NullPointerException
      	at org.apache.spark.sql.execution.datasources.v2.ShowPartitionsExec.$anonfun$run$3(ShowPartitionsExec.scala:58)
      	at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:238)
      

      Attachments

        Activity

          People

            maxgekk Max Gekk
            maxgekk Max Gekk
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: