Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-24235

Java client with IBM JDK does not work if HBase is configured with Kerberos

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Incomplete
    • 2.1.0
    • None
    • Client, java
    • None

    Description

      When a java HBase client is run with IBM JDK connecting to a HBase cluster configured with Kerberos Authentication, the client fails to connect to HBase. The client is using UGI.loginUserFromKeytab(principal, keytab)  to get a Kerberos ticket and then it is creating create connection, table, scanner and iterate. Code works fine with Oracle/Open JDK. It fails when run with IBM JDK.

      Following exception is found in logs with DEBUG level logging:

      DEBUG client.RpcRetryingCallerImpl: Call exception, tries=6, retries=11, started=4700 ms ago, cancelled=false, msg=Call to nightly6x-1.nightly6x.root.hwx.site/172.27.21.201:22101 failed on local exception: javax.security.sasl.SaslException: Failure to initialize security context [Caused by org.ietf.jgss.GSSException, major code: 13, minor code: 0
        major string: Invalid credentials
        minor string: SubjectCredFinder: no JAAS Subject], details=row 'users,,99999999999999' on table 'hbase:meta' at region=hbase:meta,,1.1588230740, hostname=nightly6x-1.nightly6x.root.hwx.site,22101,1587511170413, seqNum=-1, see https://s.apache.org/timeout, exception=javax.security.sasl.SaslException: Call to nightly6x-1.nightly6x.root.hwx.site/172.27.21.201:22101 failed on local exception: javax.security.sasl.SaslException: Failure to initialize security context [Caused by org.ietf.jgss.GSSException, major code: 13, minor code: 0
        major string: Invalid credentials
        minor string: SubjectCredFinder: no JAAS Subject] [Caused by javax.security.sasl.SaslException: Failure to initialize security context [Caused by org.ietf.jgss.GSSException, major code: 13, minor code: 0
        major string: Invalid credentials
        minor string: SubjectCredFinder: no JAAS Subject]]
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:83)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:57)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:437)
        at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:220)
        at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:390)
        at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:95)
        at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:410)
        at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:406)
        at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:103)
        at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:118)
        at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:92)
        at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326)
        at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312)
        at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:304)
        at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1426)
        at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:326)
        at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:312)
        at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:924)
        at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:179)
        at org.apache.hadoop.hbase.ipc.NettyRpcConnection.saslNegotiate(NettyRpcConnection.java:197)
        at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$800(NettyRpcConnection.java:71)
        at org.apache.hadoop.hbase.ipc.NettyRpcConnection$3.operationComplete(NettyRpcConnection.java:273)
        at org.apache.hadoop.hbase.ipc.NettyRpcConnection$3.operationComplete(NettyRpcConnection.java:261)
        at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:502)
        at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:495)
        at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:474)
        at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:415)
        at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:540)
        at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:529)
        at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:101)
        at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84)
        at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:306)
        at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:665)
        at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:612)
        at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:529)
        at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:491)
        at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905)
        at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.lang.Thread.run(Thread.java:820)
Caused by: javax.security.sasl.SaslException: Failure to initialize security context [Caused by org.ietf.jgss.GSSException, major code: 13, minor code: 0
        major string: Invalid credentials
        minor string: SubjectCredFinder: no JAAS Subject]
        at com.ibm.security.sasl.gsskerb.GssKrb5Client.<init>(GssKrb5Client.java:161)
        at com.ibm.security.sasl.gsskerb.FactoryImpl.createSaslClient(FactoryImpl.java:79)
        at javax.security.sasl.Sasl.createSaslClient(Sasl.java:400)
        at org.apache.hadoop.hbase.security.AbstractHBaseSaslRpcClient.createKerberosSaslClient(AbstractHBaseSaslRpcClient.java:125)
        at org.apache.hadoop.hbase.security.AbstractHBaseSaslRpcClient.<init>(AbstractHBaseSaslRpcClient.java:106)
        at org.apache.hadoop.hbase.security.NettyHBaseSaslRpcClient.<init>(NettyHBaseSaslRpcClient.java:43)
        at org.apache.hadoop.hbase.security.NettyHBaseSaslRpcClientHandler.<init>(NettyHBaseSaslRpcClientHandler.java:70)
        at org.apache.hadoop.hbase.ipc.NettyRpcConnection.saslNegotiate(NettyRpcConnection.java:194)
        ... 20 more
Caused by: org.ietf.jgss.GSSException, major code: 13, minor code: 0
        major string: Invalid credentials
        minor string: SubjectCredFinder: no JAAS Subject
        at com.ibm.security.jgss.i18n.I18NException.throwGSSException(Unknown Source)
        at com.ibm.security.jgss.mech.krb5.v.run(Unknown Source)
        at java.security.AccessController.doPrivileged(AccessController.java:734)
        at com.ibm.security.jgss.mech.krb5.s.c(Unknown Source)
        at com.ibm.security.jgss.mech.krb5.s.a(Unknown Source)
        at com.ibm.security.jgss.mech.krb5.s.a(Unknown Source)
        at com.ibm.security.jgss.mech.krb5.s.<init>(Unknown Source)
        at com.ibm.security.jgss.mech.krb5.Krb5MechFactory.getCredentialElement(Unknown Source)
        at com.ibm.security.jgss.GSSManagerImpl.createMechCredential(Unknown Source)
        at com.ibm.security.jgss.GSSCredentialImpl.add(Unknown Source)
        at com.ibm.security.jgss.GSSCredentialImpl.<init>(Unknown Source)
        at com.ibm.security.jgss.GSSManagerImpl.createCredential(Unknown Source)
        at com.ibm.security.jgss.GSSContextImpl.a(Unknown Source)
        at com.ibm.security.jgss.GSSContextImpl.<init>(Unknown Source)
        at com.ibm.security.jgss.GSSManagerImpl.createContext(Unknown Source)
        at com.ibm.security.sasl.gsskerb.GssKrb5Client.<init>(GssKrb5Client.java:135)
        ... 27 more

      Attachments

        Issue Links

          links to

          Web Link GitHub Pull Request #1565

          Web Link PR1565

          Activity

            People

              Unassigned Unassigned
              mkazia Mubashir Kazia
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: