Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
0.6.0
-
None
-
Reviewed
Description
Currently, if no partitions match the partition predicate, the first partition is used to retrieve the file format. This can cause an problem if the table is set to use RCFile, but the first partition uses SequenceFile:
java.lang.RuntimeException: java.lang.NoSuchMethodException: org.apache.hadoop.hive.ql.io.RCFile$KeyBuffer.() at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115) at org.apache.hadoop.mapred.SequenceFileRecordReader.createKey(SequenceFileRecordReader.java:65) at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.createKey(CombineHiveRecordReader.java:76) at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.createKey(CombineHiveRecordReader.java:42) at org.apache.hadoop.hive.shims.Hadoop20Shims$CombineFileRecordReader.createKey(Hadoop20Shims.java:212) at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.createKey(MapTask.java:167) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:45) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) at org.apache.hadoop.mapred.Child.main(Child.java:159) Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.hive.ql.io.RCFile$KeyBuffer.() at java.lang.Class.getConstructor0(Class.java:2706) at java.lang.Class.getDeclaredConstructor(Class.java:1985) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:109) ... 9 more
The proposed change is to use the table's metadata in such cases.