Details
-
Bug
-
Status: Resolved
-
Critical
-
Resolution: Fixed
-
None
-
None
-
ghx-label-2
Description
HIVE-27114 adds a new property, "hive.metastore.partitions.parameters.exclude.pattern", which defaults to "impala_intermediate_stats_chunk%". This excludes the incremental stats in the parameters returned to HMS client.
Impala is the only client that consumes this parameter. We should set it to an empty string. Otherwise, incremental stats can't be loaded when deploying Impala on newer versions of Hive. The error is
W0122 20:30:53.508314 17348 HdfsPartition.java:1444] Failed to set partition stats for table functional.alltypesagg partition year=2010/month=1/day=1
Java exception follows:
org.apache.impala.common.ImpalaRuntimeException: Missing stats chunk: 0
at org.apache.impala.catalog.PartitionStatsUtil.partStatsBytesFromParameters(PartitionStatsUtil.java:108)
at org.apache.impala.catalog.HdfsPartition$Builder.extractAndCompressPartStats(HdfsPartition.java:1441)
at org.apache.impala.catalog.HdfsPartition$Builder.setMsPartition(HdfsPartition.java:1417)
at org.apache.impala.catalog.HdfsTable.createOrUpdatePartitionBuilder(HdfsTable.java:959)
at org.apache.impala.catalog.HdfsTable.createPartitionBuilder(HdfsTable.java:948)
at org.apache.impala.catalog.HdfsTable.loadAllPartitions(HdfsTable.java:729)
at org.apache.impala.catalog.HdfsTable.load(HdfsTable.java:1299)
at org.apache.impala.catalog.HdfsTable.load(HdfsTable.java:1192)
at org.apache.impala.catalog.TableLoader.load(TableLoader.java:144)
at org.apache.impala.catalog.TableLoadingMgr$2.call(TableLoadingMgr.java:248)
at org.apache.impala.catalog.TableLoadingMgr$2.call(TableLoadingMgr.java:245)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Attachments
Issue Links
- is caused by
-
-
- Closed
-