Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-19225

Class cast exception while running certain queries with UDAF like rank on internal struct columns

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • 2.3.2
    • None
    • Hive
    • None

    Description

       

      To reproduce : [tag - 2.3.2]

      Launch hive in debug mode

      hive --hiveconf hive.root.logger=DEBUG,console;

      Run the sample sql below

      SET mapreduce.framework.name=local; 
      
      CREATE TABLE `test_class_cast` as select named_struct('a','a','b','b','c','c','d','d','e',true,'f','f','g',timestamp(1),'h','h'), 'i'; 
      
      select `_c0`.c, `_c0`.g, `_c0`.a, rank() over (partition by `_c0`.c order by `_c0`.g desc) as rown,`_c0`.f,`_c0`.e from default.test_class_cast where `_c0`.f like '%f%' or `_c0`.f like '%f%' 

      Should fail with the exception 

      Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=0) {"key":{"reducesinkkey0":"c","reducesinkkey1":"1970-01-01 05:30:00.001"},"value":{"_col0":{"a":"a","b":"b","c":"c","d":"d","e":true,"f":"f","g":"1970-01-01 05:30:00.001","h":"h"}}}
       at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.reduce(ExecReducer.java:245) ~[hive-exec-2.3.2.fk.7.jar:2.3.2.fk.7]
       at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:444) ~[hadoop-mapreduce-client-core-2.6.0.2.2.0.0-2041.jar:?]
       at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392) ~[hadoop-mapreduce-client-core-2.6.0.2.2.0.0-2041.jar:?]
       at org.apache.hadoop.mapred.LocalJobRunner$Job$ReduceTaskRunnable.run(LocalJobRunner.java:319) ~[hadoop-mapreduce-client-common-2.6.0.2.2.0.0-2041.jar:?]
       at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_92]
       at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_92]
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_92]
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_92]
       at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_92]
      Caused by: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct cannot be cast to org.apache.hadoop.hive.serde2.io.TimestampWritable
       at org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableTimestampObjectInspector.getPrimitiveJavaObject(WritableTimestampObjectInspector.java:39) ~[hive-exec-2.3.2.fk.7.jar:2.3.2.fk.7]
       at org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableTimestampObjectInspector.getPrimitiveJavaObject(WritableTimestampObjectInspector.java:25) ~[hive-exec-2.3.2.fk.7.jar:2.3.2.fk.7]
       at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject(ObjectInspectorUtils.java:412) ~[hive-exec-2.3.2.fk.7.jar:2.3.2.fk.7]
       at org.apache.hadoop.hive.ql.udf.generic.GenericUDAFRank.copyToStandardObject(GenericUDAFRank.java:219) ~[hive-exec-2.3.2.fk.7.jar:2.3.2.fk.7]

      Attachments

        1. HIVE-19225.patch
          1 kB
          Amruth Sampath

        Issue Links

          Activity

            People

              amrk7 Amruth Sampath
              amrk7 Amruth Sampath
              Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: