Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-33828 SQL Adaptive Query Execution QA
  3. SPARK-35568

UnsupportedOperationException: WholeStageCodegen (3) does not implement doExecuteBroadcast

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.2.0
    • 3.2.0
    • SQL
    • None

    Description

      How to reproduce:

      sql(
            """
              |SELECT s.store_id, f.product_id
              |FROM (SELECT DISTINCT * FROM fact_sk) f
              |  JOIN (SELECT
              |          *,
              |          ROW_NUMBER() OVER (PARTITION BY store_id ORDER BY state_province DESC) AS rn
              |        FROM dim_store) s
              |   ON f.store_id = s.store_id
              |WHERE s.country = 'DE' AND s.rn = 1
              |""".stripMargin).show
      
      Caused by: java.lang.UnsupportedOperationException: WholeStageCodegen (3) does not implement doExecuteBroadcast
      	at org.apache.spark.sql.execution.SparkPlan.doExecuteBroadcast(SparkPlan.scala:297)
      	at org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanExec.doExecuteBroadcast(AdaptiveSparkPlanExec.scala:323)
      	at org.apache.spark.sql.execution.SparkPlan.$anonfun$executeBroadcast$1(SparkPlan.scala:192)
      	at org.apache.spark.sql.execution.SparkPlan.$anonfun$executeQuery$1(SparkPlan.scala:217)
      	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
      	at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:214)
      	at org.apache.spark.sql.execution.SparkPlan.executeBroadcast(SparkPlan.scala:188)
      

      Attachments

        Activity

          People

            Jk_Self Ke Jia
            yumwang Yuming Wang
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: