Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-3002

Inconsistent accounting of exchange rows between impala-shell and profile summary

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • Impala 2.5.0
    • Impala 2.8.0
    • Clients
    • Bolt-80

    Description

      Check 04:EXCHANGE

      Summary from impala-shell

      Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail
      06:AGGREGATE 1 201.91ms 201.91ms 1 1 300.00 KB -1 B FINALIZE
      05:EXCHANGE 1 1.28ms 1.28ms 72 1 0 B -1 B UNPARTITIONED
      03:AGGREGATE 72 5.01s 6.47s 72 1 3.22 MB 10.00 MB  
      02:HASH JOIN 72 203.42s 262.39s 3.26B 6.36B 10.51 GB 11.63 GB INNER JOIN, BROADCAST
      I--04:EXCHANGE 72 13.07s 14.28s 7.83B 200.00M 0 B 0 B BROADCAST
      I 00:SCAN HDFS 72 550.48ms 634.76ms 108.70M 200.00M 476.01 MB 176.00 MB tpch_10000_decimal_parquet.part
      01:SCAN HDFS 72 4.64s 7.13s 60.00B 60.00B 1.18 GB 352.00 MB tpch_10000_decimal_parquet.lineitem

      And from profiles

      Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail
      06:AGGREGATE 1 201.907ms 201.907ms 1 1 300.00 KB -1.00 B FINALIZE
      05:EXCHANGE 1 1.284ms 1.284ms 72 1 0 -1.00 B UNPARTITIONED
      03:AGGREGATE 72 5s011ms 6s467ms 72 1 3.22 MB 10.00 MB
      02:HASH JOIN 72 3m23s 4m22s 3.26B 6.36B 10.51 GB 11.63 GB INNER JOIN, BROADCAST
      I--04:EXCHANGE 72 13s070ms 14s275ms 108.70M 200.00M 0 0 BROADCAST
      I 00:SCAN HDFS 72 550.477ms 634.764ms 108.70M 200.00M 476.01 MB 176.00 MB tpch_10000_decimal_parquet....
      01:SCAN HDFS 72 4s641ms 7s125ms 60.00B 60.00B 1.18 GB 352.00 MB tpch_10000_decimal_parquet....

      Attachments

        Activity

          People

            twmarshall Thomas Tauber-Marshall
            mmokhtar Mostafa Mokhtar
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: