Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-28106

TestShadeSaslAuthenticationProvider fails for branch-2.x

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • None
    • 2.6.0, 2.4.18, 2.5.6
    • test
    • None
    • Reviewed

    Description

      org.apache.hadoop.hbase.security.provider.example.TestShadeSaslAuthenticationProvider is failing consistently in branch-2.x

      This could be a code bug as well!

       

      For branch-2:

      For branch-2.5

      For branch-2.4

       

      StackTrace copied from failed test case:

      java.lang.Exception: Unexpected exception, expected<org.apache.hadoop.hbase.DoNotRetryIOException> but was<org.apache.hadoop.hbase.client.RetriesExhaustedException>
      	at org.junit.internal.runners.statements.ExpectException.evaluate(ExpectException.java:30)
      	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
      	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
      	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
      	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
      	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
      	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
      	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
      	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
      	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
      	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
      	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
      	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
      	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
      	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
      	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
      	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
      	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
      	at java.base/java.lang.Thread.run(Thread.java:829)
      Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=4, exceptions:
      2023-09-21T17:53:07.490Z, RpcRetryingCaller{globalStartTime=2023-09-21T17:52:04.943Z, pause=100, maxAttempts=4}, org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=4, exceptions:
      2023-09-21T17:53:06.869Z, RpcRetryingCaller{globalStartTime=2023-09-21T17:53:06.862Z, pause=100, maxAttempts=4}, org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Call to address=6a87b11f0f42:38383 failed on local exception: org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Connection closed
      2023-09-21T17:53:06.975Z, RpcRetryingCaller{globalStartTime=2023-09-21T17:53:06.862Z, pause=100, maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383 failed on local exception: java.io.IOException: Connection reset by peer
      2023-09-21T17:53:07.182Z, RpcRetryingCaller{globalStartTime=2023-09-21T17:53:06.862Z, pause=100, maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383 failed on local exception: java.io.IOException: Connection reset by peer
      2023-09-21T17:53:07.489Z, RpcRetryingCaller{globalStartTime=2023-09-21T17:53:06.862Z, pause=100, maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383 failed on local exception: java.io.IOException: Connection reset by peer
      
      2023-09-21T17:53:08.215Z, RpcRetryingCaller{globalStartTime=2023-09-21T17:52:04.943Z, pause=100, maxAttempts=4}, org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=4, exceptions:
      2023-09-21T17:53:07.597Z, RpcRetryingCaller{globalStartTime=2023-09-21T17:53:07.591Z, pause=100, maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383 failed on local exception: java.io.IOException: Connection reset by peer
      2023-09-21T17:53:07.703Z, RpcRetryingCaller{globalStartTime=2023-09-21T17:53:07.591Z, pause=100, maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383 failed on local exception: java.io.IOException: Connection reset by peer
      2023-09-21T17:53:07.909Z, RpcRetryingCaller{globalStartTime=2023-09-21T17:53:07.591Z, pause=100, maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383 failed on local exception: java.io.IOException: Connection reset by peer
      2023-09-21T17:53:08.215Z, RpcRetryingCaller{globalStartTime=2023-09-21T17:53:07.591Z, pause=100, maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383 failed on local exception: java.io.IOException: Connection reset by peer
      
      2023-09-21T17:53:09.057Z, RpcRetryingCaller{globalStartTime=2023-09-21T17:52:04.943Z, pause=100, maxAttempts=4}, org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=4, exceptions:
      2023-09-21T17:53:08.422Z, RpcRetryingCaller{globalStartTime=2023-09-21T17:53:08.416Z, pause=100, maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383 failed on local exception: java.io.IOException: Connection reset by peer
      2023-09-21T17:53:08.530Z, RpcRetryingCaller{globalStartTime=2023-09-21T17:53:08.416Z, pause=100, maxAttempts=4}, org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Call to address=6a87b11f0f42:38383 failed on local exception: org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Connection closed
      2023-09-21T17:53:08.748Z, RpcRetryingCaller{globalStartTime=2023-09-21T17:53:08.416Z, pause=100, maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383 failed on local exception: java.io.IOException: Connection reset by peer
      2023-09-21T17:53:09.056Z, RpcRetryingCaller{globalStartTime=2023-09-21T17:53:08.416Z, pause=100, maxAttempts=4}, org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Call to address=6a87b11f0f42:38383 failed on local exception: org.apache.hadoop.hbase.exceptions.ConnectionClosedException: Connection closed
      
      2023-09-21T17:53:09.990Z, RpcRetryingCaller{globalStartTime=2023-09-21T17:52:04.943Z, pause=100, maxAttempts=4}, org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=4, exceptions:
      2023-09-21T17:53:09.362Z, RpcRetryingCaller{globalStartTime=2023-09-21T17:53:09.357Z, pause=100, maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383 failed on local exception: java.io.IOException: Connection reset by peer
      2023-09-21T17:53:09.469Z, RpcRetryingCaller{globalStartTime=2023-09-21T17:53:09.357Z, pause=100, maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383 failed on local exception: java.io.IOException: Connection reset by peer
      2023-09-21T17:53:09.683Z, RpcRetryingCaller{globalStartTime=2023-09-21T17:53:09.357Z, pause=100, maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383 failed on local exception: java.io.IOException: Connection reset by peer
      2023-09-21T17:53:09.990Z, RpcRetryingCaller{globalStartTime=2023-09-21T17:53:09.357Z, pause=100, maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383 failed on local exception: java.io.IOException: Connection reset by peer
      
      
      	at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:141)
      	at org.apache.hadoop.hbase.client.HTable.get(HTable.java:390)
      	at org.apache.hadoop.hbase.client.HTable.lambda$get$0(HTable.java:363)
      	at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216)
      	at org.apache.hadoop.hbase.client.HTable.get(HTable.java:363)
      	at org.apache.hadoop.hbase.security.provider.example.TestShadeSaslAuthenticationProvider$3.run(TestShadeSaslAuthenticationProvider.java:235)
      	at org.apache.hadoop.hbase.security.provider.example.TestShadeSaslAuthenticationProvider$3.run(TestShadeSaslAuthenticationProvider.java:230)
      	at java.base/java.security.AccessController.doPrivileged(Native Method)
      	at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
      	at org.apache.hadoop.hbase.security.provider.example.TestShadeSaslAuthenticationProvider.testNegativeAuthentication(TestShadeSaslAuthenticationProvider.java:230)
      	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
      	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
      	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
      	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
      	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
      	at org.junit.internal.runners.statements.ExpectException.evaluate(ExpectException.java:19)
      	... 18 more
      Caused by: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=4, exceptions:
      2023-09-21T17:53:09.362Z, RpcRetryingCaller{globalStartTime=2023-09-21T17:53:09.357Z, pause=100, maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383 failed on local exception: java.io.IOException: Connection reset by peer
      2023-09-21T17:53:09.469Z, RpcRetryingCaller{globalStartTime=2023-09-21T17:53:09.357Z, pause=100, maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383 failed on local exception: java.io.IOException: Connection reset by peer
      2023-09-21T17:53:09.683Z, RpcRetryingCaller{globalStartTime=2023-09-21T17:53:09.357Z, pause=100, maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383 failed on local exception: java.io.IOException: Connection reset by peer
      2023-09-21T17:53:09.990Z, RpcRetryingCaller{globalStartTime=2023-09-21T17:53:09.357Z, pause=100, maxAttempts=4}, java.io.IOException: Call to address=6a87b11f0f42:38383 failed on local exception: java.io.IOException: Connection reset by peer
      
      	at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:141)
      	at org.apache.hadoop.hbase.client.HTable.get(HTable.java:390)
      	at org.apache.hadoop.hbase.client.HTable.lambda$get$0(HTable.java:363)
      	at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:216)
      	at org.apache.hadoop.hbase.client.HTable.get(HTable.java:363)
      	at org.apache.hadoop.hbase.MetaTableAccessor.getTableState(MetaTableAccessor.java:1141)
      	at org.apache.hadoop.hbase.client.ConnectionImplementation.getTableState(ConnectionImplementation.java:2235)
      	at org.apache.hadoop.hbase.client.ConnectionImplementation.isTableDisabled(ConnectionImplementation.java:742)
      	at org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:214)
      	at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:101)
      	... 37 more
      Caused by: java.io.IOException: Call to address=6a87b11f0f42:38383 failed on local exception: java.io.IOException: Connection reset by peer
      	at java.base/jdk.internal.reflect.GeneratedConstructorAccessor69.newInstance(Unknown Source)
      	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
      	at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:239)
      	at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:391)
      	at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:92)
      	at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:425)
      	at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:420)
      	at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:114)
      	at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:129)
      	at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.cleanupCalls(NettyRpcDuplexHandler.java:212)
      	at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.exceptionCaught(NettyRpcDuplexHandler.java:229)
      	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346)
      	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:325)
      	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:317)
      	at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377)
      	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346)
      	at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:325)
      	at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907)
      	at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.handleReadException(AbstractNioByteChannel.java:125)
      	at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:177)
      	at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
      	at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
      	at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
      	at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
      	at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
      	at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
      	at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
      	... 1 more
      Caused by: java.io.IOException: Connection reset by peer
      	at java.base/sun.nio.ch.FileDispatcherImpl.read0(Native Method)
      	at java.base/sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
      	at java.base/sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:276)
      	at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:233)
      	at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:223)
      	at java.base/sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:356)
      	at org.apache.hbase.thirdparty.io.netty.buffer.PooledByteBuf.setBytes(PooledByteBuf.java:259)
      	at org.apache.hbase.thirdparty.io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1132)
      	at org.apache.hbase.thirdparty.io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:357)
      	at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:151)
      	... 8 more
      
      

       
      Also note the test class code is not same between master and branch-2.x

      This issue could be related to HBASE-28101

      Attachments

        Issue Links

          Activity

            People

              zhangduo Duo Zhang
              nihaljain.cs Nihal Jain
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: