Region server abort with failing to list region servers.

During unit tests, clusters were created and destroyed, the region server could abort during shutdown with failing to list region servers.

 

Logs:

 

2022-10-11 18:44:43,393 INFO  [main] utility.HBaseClusterSetup(160): Starting mini cluster: singleCluster

[master/victorli-wsl:0:becomeActiveMaster] INFO org.apache.hadoop.hbase.zookeeper.ZKWatcher - not a secure deployment, proceeding

2022-10-11 18:45:07,234 INFO  [Listener at localhost/45185] utility.HBaseClusterSetup(177): Started mini cluster: singleCluster

[Listener at localhost/45185] INFO org.apache.hadoop.hbase.zookeeper.ZKWatcher - ZKWatcher close started.

[Listener at localhost/45185] INFO org.apache.hadoop.hbase.zookeeper.ZKWatcher - ZKWatcher close finished.

[Listener at localhost/45185] INFO org.apache.hadoop.hbase.zookeeper.ZKWatcher - ZKWatcher close started.

[Listener at localhost/45185] INFO org.apache.hadoop.hbase.zookeeper.ZKWatcher - ZKWatcher close finished.

2022-10-11 18:45:45,980 INFO  [Listener at localhost/45185] utility.HBaseClusterSetup(132): Stopping mini cluster: singleCluster

[RS:0;victorli-wsl:35113] INFO org.apache.hadoop.hbase.zookeeper.ZKWatcher - ZKWatcher close started.

[zk-event-processor-pool-0] ERROR org.apache.hadoop.hbase.regionserver.HRegionServer - ***** ABORTING region server victorli-wsl,35113,1665539090704: failed to list region servers *****

[RS:0;victorli-wsl:35113] INFO org.apache.hadoop.hbase.zookeeper.ZKWatcher - ZKWatcher close finished.

[M:0;victorli-wsl:42915] INFO org.apache.hadoop.hbase.zookeeper.ZKWatcher - ZKWatcher close started.

[M:0;victorli-wsl:42915] INFO org.apache.hadoop.hbase.zookeeper.ZKWatcher - ZKWatcher close finished.

2022-10-11 18:45:57,208 INFO  [Listener at localhost/45185] utility.HBaseClusterSetup(144): Stopped mini cluster: singleCluster

 

Stack:

[zk-event-processor-pool-0] WARN org.apache.hadoop.hbase.zookeeper.ZKUtil - regionserver:35113-0x101b6fed2a00001, quorum=127.0.0.1:53028, baseZNode=/9 Unable to list children of znode /9/rs

java.lang.InterruptedException

    at java.lang.Object.wait(Native Method)

    at java.lang.Object.wait(Object.java:502)

    at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1601)

    at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1573)

    at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:2420)

    at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getChildren(RecoverableZooKeeper.java:341)

    at org.apache.hadoop.hbase.zookeeper.ZKUtil.listChildrenAndWatchForNewChildren(ZKUtil.java:190)

    at org.apache.hadoop.hbase.zookeeper.RegionServerAddressTracker.loadRegionServerList(RegionServerAddressTracker.java:54)

    at org.apache.hadoop.hbase.zookeeper.RegionServerAddressTracker.nodeChildrenChanged(RegionServerAddressTracker.java:71)

    at org.apache.hadoop.hbase.zookeeper.ZKWatcher.lambda$processEvent$0(ZKWatcher.java:586)

    at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:187)

    at org.apache.hadoop.hbase.trace.TraceUtil.trace(TraceUtil.java:177)

    at org.apache.hadoop.hbase.zookeeper.ZKWatcher.processEvent(ZKWatcher.java:554)

    at org.apache.hadoop.hbase.zookeeper.ZKWatcher.lambda$process$1(ZKWatcher.java:607)

    at org.apache.hadoop.hbase.trace.TraceUtil.lambda$tracedRunnable$2(TraceUtil.java:155)

    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

    at java.lang.Thread.run(Thread.java:750)