Details
-
Bug
-
Status: Resolved
-
Critical
-
Resolution: Fixed
-
None
-
None
-
ghx-label-2
Description
Concurrent invalidates with partial RPC on partitioned tables can throw this exception.
I1016 15:49:03.438048 30197 jni-util.cc:256] java.lang.IllegalArgumentException: Partition id 162 does not exist at com.google.common.base.Preconditions.checkArgument(Preconditions.java:119) at org.apache.impala.catalog.HdfsTable.getPartialInfo(HdfsTable.java:1711) at org.apache.impala.catalog.CatalogServiceCatalog.doGetPartialCatalogObject(CatalogServiceCatalog.java:2202) at org.apache.impala.catalog.CatalogServiceCatalog.getPartialCatalogObject(CatalogServiceCatalog.java:2141) at org.apache.impala.service.JniCatalog.getPartialCatalogObject(JniCatalog.java:237) I1016 15:49:03.440939 30197 status.cc:129] IllegalArgumentException: Partition id 162 does not exist
@Override
public TGetPartialCatalogObjectResponse getPartialInfo(
TGetPartialCatalogObjectRequest req) throws TableLoadingException {
if (partIds != null) {
resp.table_info.partitions = Lists.newArrayListWithCapacity(partIds.size());
for (long partId : partIds) {
HdfsPartition part = partitionMap_.get(partId);
Preconditions.checkArgument(part != null, "Partition id %s does not exist",
partId); <====
The issue is that the invalidate command can reset the partition IDs and the RPCs could look up with older IDs.
We should wrap this into an inconsistent metadata fetch exception and retry rather than throwing a RTE.