Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-46295

TPCDS q39a and a39b have correctness issues with broadcast hash join and shuffled hash join

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.4.2, 3.5.0, 4.0.0
    • None
    • SQL

    Description

      TPCDSQueryTestSuite fails for q39a and a39b with broadcastHashJoinConf and shuffledHashJoinConf. It works fine with sortMergeJoinConf

      SPARK_TPCDS_DATA=<path of TPCDS SF=1 data> build/sbt "~sql/testOnly *TPCDSQueryTestSuite -- -z q39a"
      [info] - q39a *** FAILED *** (19 seconds, 139 milliseconds)
      [info]   java.lang.Exception: Expected "...25	1.022382911080458[8 ..." but got "...25	1.022382911080458[5 ..."
      
      SPARK_TPCDS_DATA=<path of TPCDS SF=1 data> build/sbt "~sql/testOnly *TPCDSQueryTestSuite -- -z q39b"
      [info] - q39b *** FAILED *** (19 seconds, 351 milliseconds)
      [info]   java.lang.Exception: Expected "...34	1.563403519178623[3	3	10427	2	381.25	1.0623056061004696
      [info] 3	3315	1	271.75	1.555976998814345	3	3315	2	393.75	1.0196319345405949
      [info] 3	3393	1	260.0	1.5009563026568116	3	3393	2	470.25	1.129275872154205
      [info] 4	16211	1	257.6666666666667	1.6381074811154002]	4	16211	2	352.25	1....", but got "...34	1.563403519178623[5	3	10427	2	381.25	1.0623056061004696
      [info] 3	3315	1	271.75	1.555976998814345	3	3315	2	393.75	1.0196319345405949
      [info] 3	3393	1	260.0	1.5009563026568118	3	3393	2	470.25	1.129275872154205
      [info] 4	16211	1	257.6666666666667	1.6381074811154]	4	16211	2	352.25	1...." Result did not match
      

      Attachments

        Activity

          People

            Unassigned Unassigned
            kazuyukitanimura Kazuyuki Tanimura
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: