Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-3803

Fix Hive3.1.3 Metastore service compatible issue with Hadoop3.3.x when kerberos enabled

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.2.0
    • None
    • None
    • Bigtop-3.2.0-snapshot

    Description

      Fail to start Hive metastore when enable Kerberos in Hadoop 3.3.x

      2022-09-02T06:20:04,458 INFO  [main]: zookeeper.ZooKeeper (ZooKeeper.java:<init>(868)) - Initiating client connection, connectString=ambari-agent-01:2181,ambari-agent-02:2181,ambari-server:2181 sessionTimeout=60000 watcher=org.apache.curator.ConnectionState@433348bc
      2022-09-02T06:20:04,470 INFO  [main]: common.X509Util (X509Util.java:<clinit>(79)) - Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation
      2022-09-02T06:20:04,499 INFO  [main]: zookeeper.ClientCnxnSocket (ClientCnxnSocket.java:initProperties(237)) - jute.maxbuffer value is 4194304 Bytes
      2022-09-02T06:20:04,540 INFO  [main]: zookeeper.ClientCnxn (ClientCnxn.java:initRequestTimeout(1653)) - zookeeper.request.timeout value is 0. feature enabled=
      2022-09-02T06:20:04,587 ERROR [main]: metastore.HiveMetaStore (HiveMetaStore.java:startMetaStore(9056)) - java.lang.NoSuchMethodError: org.apache.curator.framework.api.CreateBuilder.creatingParentsIfNeeded()Lorg/apache/curator/framework/api/ProtectACLCreateModeStatPathAndBytesable;
      	at org.apache.hadoop.hive.metastore.security.ZooKeeperTokenStore.ensurePath(ZooKeeperTokenStore.java:158)
      	at org.apache.hadoop.hive.metastore.security.ZooKeeperTokenStore.initClientAndPaths(ZooKeeperTokenStore.java:234)
      	at org.apache.hadoop.hive.metastore.security.ZooKeeperTokenStore.init(ZooKeeperTokenStore.java:471)
      	at org.apache.hadoop.hive.metastore.security.MetastoreDelegationTokenManager.startDelegationTokenSecretManager(MetastoreDelegationTokenManager.java:76)
      	at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:8952)
      	at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:8854)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
      	at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
      
      2022-09-02T06:20:04,613 INFO  [main-SendThread(ambari-agent-01:2181)]: zookeeper.Login (Login.java:login(302)) - HiveZooKeeperClient successfully logged in.
      2022-09-02T06:20:04,615 INFO  [Thread-6]: zookeeper.Login (Login.java:run(135)) - TGT refresh thread started.
      2022-09-02T06:20:04,640 INFO  [main-SendThread(ambari-agent-01:2181)]: client.ZooKeeperSaslClient (SecurityUtils.java:run(128)) - Client will use GSSAPI as SASL mechanism.
      2022-09-02T06:20:04,588 ERROR [main]: metastore.HiveMetaStore (HiveMetaStore.java:main(8859)) - Metastore Thrift Server threw an exception...
      java.lang.NoSuchMethodError: org.apache.curator.framework.api.CreateBuilder.creatingParentsIfNeeded()Lorg/apache/curator/framework/api/ProtectACLCreateModeStatPathAndBytesable;
      	at org.apache.hadoop.hive.metastore.security.ZooKeeperTokenStore.ensurePath(ZooKeeperTokenStore.java:158) ~[hive-exec-3.1.3.jar:3.1.3]
      	at org.apache.hadoop.hive.metastore.security.ZooKeeperTokenStore.initClientAndPaths(ZooKeeperTokenStore.java:234) ~[hive-exec-3.1.3.jar:3.1.3]
      	at org.apache.hadoop.hive.metastore.security.ZooKeeperTokenStore.init(ZooKeeperTokenStore.java:471) ~[hive-exec-3.1.3.jar:3.1.3]
      	at org.apache.hadoop.hive.metastore.security.MetastoreDelegationTokenManager.startDelegationTokenSecretManager(MetastoreDelegationTokenManager.java:76) ~[hive-exec-3.1.3.jar:3.1.3]
      	at org.apache.hadoop.hive.metastore.HiveMetaStore.startMetaStore(HiveMetaStore.java:8952) ~[hive-exec-3.1.3.jar:3.1.3]
      	at org.apache.hadoop.hive.metastore.HiveMetaStore.main(HiveMetaStore.java:8854) [hive-exec-3.1.3.jar:3.1.3]
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_332]
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_332]
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_332]
      	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_332]
      	at org.apache.hadoop.util.RunJar.run(RunJar.java:323) [hadoop-common-3.3.4.jar:?]
      	at org.apache.hadoop.util.RunJar.main(RunJar.java:236) [hadoop-common-3.3.4.jar:?]
      
      

      Attachments

        Issue Links

          Activity

            People

              houyu Yu Hou
              houyu Yu Hou
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h 40m
                  2h 40m