Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
Impala 2.6.0
-
None
-
Workaround: resolve the kerberos authentication problem then run "invalidate metadata <table name>" on the affected table.
Description
We've seen a transient kerberos error cause a table to get into a bad state until the metadata is force-reloaded by "invalidate metadata". We should consider improving the error message or automatically retry reloading the table after some period so that the transient failure remains transient.
CAUSED BY: TableLoadingException: Failed to load metadata for table: foo.bar. Running 'invalidate foo.bar' may resolve this problem. CAUSED BY: IllegalStateException: MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: GSS initiate failed at org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:221) at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:297) at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37) at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52) at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1709) at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:430) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:240) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:185) at com.cloudera.impala.catalog.MetaStoreClientPool$MetaStoreClient.<init>(MetaStoreClientPool.java:57) at com.cloudera.impala.catalog.MetaStoreClientPool$MetaStoreClient.<init>(MetaStoreClientPool.java:50) at com.cloudera.impala.catalog.MetaStoreClientPool.getClient(MetaStoreClientPool.java:145) at com.cloudera.impala.catalog.Catalog.getMetaStoreClient(Catalog.java:320) at com.cloudera.impala.catalog.TableLoader.load(TableLoader.java:61) at com.cloudera.impala.catalog.TableLoadingMgr$2.call(TableLoadingMgr.java:229) at com.cloudera.impala.catalog.TableLoadingMgr$2.call(TableLoadingMgr.java:226)