Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-9132 CBO: Calcite Operator To Hive Operator (Calcite Return Path)
  3. HIVE-10463

CBO (Calcite Return Path): Insert overwrite... select * from... queries failing for bucketed tables

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.3.0
    • CBO
    • None

    Description

      When return path is on. To reproduce the Exception, take the following excerpt from auto_sortmerge_join_10.q:

      set hive.enforce.bucketing = true;
      set hive.enforce.sorting = true;
      set hive.exec.reducers.max = 1;
      
      CREATE TABLE tbl1(key int, value string) CLUSTERED BY (key) SORTED BY (key) INTO 2 BUCKETS;
      
      insert overwrite table tbl1
      select * from src where key < 10;
      

      It produces the following Exception:

      java.lang.Exception: java.lang.RuntimeException: Error in configuring object
              at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
              at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:529)
      Caused by: java.lang.RuntimeException: Error in configuring object
              at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
              at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
              at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
              at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:409)
              at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
              at org.apache.hadoop.mapred.LocalJobRunner$Job$ReduceTaskRunnable.run(LocalJobRunner.java:319)
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
              at java.util.concurrent.FutureTask.run(FutureTask.java:262)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
              at java.lang.Thread.run(Thread.java:745)
      Caused by: java.lang.reflect.InvocationTargetException
              at sun.reflect.GeneratedMethodAccessor16.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:606)
              at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
              ... 10 more
      Caused by: java.lang.RuntimeException: Reduce operator initialization failed
              at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.configure(ExecReducer.java:157)
              ... 14 more
      Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: cannot find field key from [0:_col0, 1:_col1]
              at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:446)
              at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:362)
              at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:481)
              at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:438)
              at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
              at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:481)
              at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:438)
              at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
              at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.configure(ExecReducer.java:150)
              ... 14 more
      Caused by: java.lang.RuntimeException: cannot find field key from [0:_col0, 1:_col1]
              at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:416)
              at org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.getStructFieldRef(StandardStructObjectInspector.java:147)
              at org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:55)
              at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:978)
              at org.apache.hadoop.hive.ql.exec.FileSinkOperator.initializeOp(FileSinkOperator.java:383)
              ... 22 more
      

      Attachments

        Activity

          People

            jpullokkaran Laljo John Pullokkaran
            jcamacho Jesús Camacho Rodríguez
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: