Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-10484

Vectorization : RuntimeException "Big Table Retained Mapping duplicate column"

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.2.0
    • 1.2.0
    • Tez, Vectorization
    • None

    Description

      With vectorization and tez enabled TPC-DS Q70 fails with

      Caused by: java.lang.RuntimeException: Big Table Retained Mapping duplicate column 6 in ordered column map {6=(value column: 6, type name: int), 21=(value column: 21, type name: float), 22=(value column: 22, type name: int)} when adding value column 6, type int
      	at org.apache.hadoop.hive.ql.exec.vector.VectorColumnOrderedMap.add(VectorColumnOrderedMap.java:97)
      	at org.apache.hadoop.hive.ql.exec.vector.VectorColumnOutputMapping.add(VectorColumnOutputMapping.java:40)
      	at org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinCommonOperator.determineCommonInfo(VectorMapJoinCommonOperator.java:320)
      	at org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinCommonOperator.<init>(VectorMapJoinCommonOperator.java:254)
      	at org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinGenerateResultOperator.<init>(VectorMapJoinGenerateResultOperator.java:89)
      	at org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerGenerateResultOperator.<init>(VectorMapJoinInnerGenerateResultOperator.java:97)
      	at org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerLongOperator.<init>(VectorMapJoinInnerLongOperator.java:79)
      	... 49 more
      

      Query

       select s_state
                     from  (select s_state as s_state, sum(ss_net_profit),
                                   rank() over ( partition by s_state order by sum(ss_net_profit) desc) as ranking
                            from   store_sales, store, date_dim
                            where  d_month_seq between 1193 and 1193+11
                                  and date_dim.d_date_sk = store_sales.ss_sold_date_sk
                                  and store.s_store_sk  = store_sales.ss_store_sk
                            group by s_state
                           ) tmp1
                     where ranking <= 5
      

      Attachments

        1. HIVE-10484.01.patch
          1.0 kB
          Matt McCline
        2. HIVE-10484.02.patch
          1.0 kB
          Sushanth Sowmyan

        Activity

          People

            mmccline Matt McCline
            mmokhtar Mostafa Mokhtar
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: