Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
1.0.0
-
None
Description
Region will not be reassigned if encounters ConnectionClosingException, and then it will stuck in PENDING_OPEN state. Error logs are as follows,
INFO [jd-data-hbase02.gh.sankuai.com,16000,1621944138744-GeneralBulkAssigner-12] master.AssignmentManager: Unable to communicate with jd-data-hbase15.gh.sankuai.com,16020,1622026221268 in order to assign regions, org.apache.hadoop.hbase.exceptions.ConnectionClosingException: Call to jd-data-hbase15.gh.sankuai.com/10.78.96.166:16020 failed on local exception: org.apache.hadoop.hbase.exceptions.ConnectionClosingException: Connection to jd-data-hbase15.gh.sankuai.com/10.78.96.166:16020 is closing. Call id=19239, waitTime=1 at org.apache.hadoop.hbase.ipc.AbstractRpcClient.wrapException(AbstractRpcClient.java:289) at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1270) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:227) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:336) at org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$BlockingStub.openRegion(AdminProtos.java:25890) at org.apache.hadoop.hbase.master.ServerManager.sendRegionOpen(ServerManager.java:798) at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1744) at org.apache.hadoop.hbase.master.GeneralBulkAssigner$SingleServerBulkAssigner.run(GeneralBulkAssigner.java:203) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.hadoop.hbase.exceptions.ConnectionClosingException: Connection to jd-data-hbase15.gh.sankuai.com/10.78.96.166:16020 is closing. Call id=19239, waitTime=1 at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.cleanupCalls(RpcClientImpl.java:1083) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.close(RpcClientImpl.java:863) at org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.run(RpcClientImpl.java:580)
Attachments
Issue Links
- links to