Details
-
Bug
-
Status: Resolved
-
Blocker
-
Resolution: Fixed
-
Impala 3.1.0
-
ghx-label-7
Description
A build on latest master failed due to a Hive Metastore failure:
09:40:14 Formatting cluster
09:40:14 Killing running services...
...
09:40:17 Starting cluster services...
...
09:40:35 The cluster is running
09:40:35 --> Starting Hive Metastore Service
...
09:45:36 Waiting for the Metastore at localhost:9083...
09:45:36 Metastore service failed to start within 300.0 seconds.
09:45:36 ERROR in /data/jenkins/workspace/impala-cdh6.x-core-s3/repos/Impala/testdata/bin/run-hive-server.sh at line 75: ${CLUSTER_BIN}/wait-for-metastore.py --transport=${METASTORE_TRANSPORT}
The hive-metastore.out file suggests a problem with the Metastore DB:
19/01/25 09:40:44 INFO DataNucleus.Persistence: Property hive.server2.idle.session.timeout unknown - will be ignored
javax.jdo.JDODataStoreException: Exception thrown flushing changes to datastore
at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
at org.datanucleus.api.jdo.JDOTransaction.commit(JDOTransaction.java:171)
at org.apache.hadoop.hive.metastore.ObjectStore.commitTransaction(ObjectStore.java:729)
at org.apache.hadoop.hive.metastore.ObjectStore.createDatabase(ObjectStore.java:786)
...
NestedThrowablesStackTrace:
java.sql.BatchUpdateException: Batch entry 0 INSERT INTO "DBS" ("DB_ID","CREATE_TIME","DESC","DB_LOCATION_URI","NAME","OWNER_NAME","OWNER_TYPE") VALUES ('687','1548438048','Default Hive database','s3a://impala-test-uswest2-2/test-warehouse','default','public','ROLE') was aborted. Call getNextException to see the cause.
Could not find (probably don't know where to look) errors from the underlying DB.