Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-22805

Vectorization with conditional array or map is not implemented and throws an error

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 4.0.0-alpha-1
    • Vectorization
    • None

    Description

      The following queries throw an "Not supported" Exception:

      DROP TABLE IF EXISTS cond_vector;
      CREATE TABLE cond_vector(a STRING) STORED AS ORC;
      INSERT OVERWRITE TABLE cond_vector VALUES("a/b");
      set hive.fetch.task.conversion=minimal;
      set hive.execution.mode=container;
      
      SELECT IF(1=1, MAP("a","b"), NULL) FROM cond_vector;
      EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, MAP("Mathematics","78"), NULL) FROM cond_vector;
      
      SELECT IF(1=1, ARRAY("c", "d"), NULL) FROM cond_vector;
      EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, ARRAY("a", "b"), NULL) FROM cond_vector;
      

      One example of the Exception (maybe not exactly this on 4.0.0 branch, but close enough to check):

      Caused by: java.lang.RuntimeException: Not supported
      	at org.apache.hadoop.hive.ql.exec.vector.ListColumnVector.copySelected(ListColumnVector.java:161)
      	at org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprNull.evaluate(IfExprCondExprNull.java:87)
      	at org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:95)
      	at org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprBase.conditionalEvaluate(IfExprCondExprBase.java:68)
      	at org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:110)
      	at org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:95)
      	at org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:146) 
      

      Attachments

        1. HIVE-22805.3.patch
          38 kB
          Peter Vary
        2. HIVE-22805.2.patch
          15 kB
          Peter Vary
        3. HIVE-22805.patch
          15 kB
          Peter Vary

        Issue Links

          Activity

            People

              pvary Peter Vary
              pvary Peter Vary
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: