Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
while investigating an (unrelated) test bug in CollectionPropsTest I discovered a deadlock situation that can occur when calling ZkStateReader.removeCollectionPropsWatcher() if a zkCallback thread tries to concurrently fire the watchers set on the collection props.
ZkStateReader.removeCollectionPropsWatcher() is itself called when a CollectionPropsWatcher.onStateChanged() impl returns "true" – meaning that IIUC any usage of CollectionPropsWatcher could potentially result in this type of deadlock situation.
"TEST-CollectionPropsTest.testReadWriteCached-seed#[D3C6921874D1CFEB]" #15 prio=5 os_prio=0 cpu=567.78ms elapsed=682.12s tid=0x00007
fa5e8343800 nid=0x3f61 waiting for monitor entry [0x00007fa62d222000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.apache.solr.common.cloud.ZkStateReader.lambda$removeCollectionPropsWatcher$20(ZkStateReader.java:2001)
- waiting to lock <0x00000000e6207500> (a java.util.concurrent.ConcurrentHashMap)
at org.apache.solr.common.cloud.ZkStateReader$$Lambda$617/0x00000001006c1840.apply(Unknown Source)
at java.util.concurrent.ConcurrentHashMap.compute(java.base@11.0.3/ConcurrentHashMap.java:1932)
- locked <0x00000000eb9156b8> (a java.util.concurrent.ConcurrentHashMap$Node)
at org.apache.solr.common.cloud.ZkStateReader.removeCollectionPropsWatcher(ZkStateReader.java:1994)
at org.apache.solr.cloud.CollectionPropsTest.testReadWriteCached(CollectionPropsTest.java:125)
...
"zkCallback-88-thread-2" #213 prio=5 os_prio=0 cpu=14.06ms elapsed=672.65s tid=0x00007fa6041bf000 nid=0x402f waiting for monitor ent
ry [0x00007fa5b8f39000]
java.lang.Thread.State: BLOCKED (on object monitor)
at java.util.concurrent.ConcurrentHashMap.compute(java.base@11.0.3/ConcurrentHashMap.java:1923)
- waiting to lock <0x00000000eb9156b8> (a java.util.concurrent.ConcurrentHashMap$Node)
at org.apache.solr.common.cloud.ZkStateReader$PropsNotification.<init>(ZkStateReader.java:2262)
at org.apache.solr.common.cloud.ZkStateReader.notifyPropsWatchers(ZkStateReader.java:2243)
at org.apache.solr.common.cloud.ZkStateReader$PropsWatcher.refreshAndWatch(ZkStateReader.java:1458)
- locked <0x00000000e6207500> (a java.util.concurrent.ConcurrentHashMap)
at org.apache.solr.common.cloud.ZkStateReader$PropsWatcher.process(ZkStateReader.java:1440)
at org.apache.solr.common.cloud.SolrZkClient$ProcessWatchWithExecutor.lambda$process$1(SolrZkClient.java:838)
at org.apache.solr.common.cloud.SolrZkClient$ProcessWatchWithExecutor$$Lambda$253/0x00000001004a4440.run(Unknown Source)
at java.util.concurrent.Executors$RunnableAdapter.call(java.base@11.0.3/Executors.java:515)
at java.util.concurrent.FutureTask.run(java.base@11.0.3/FutureTask.java:264)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$140/0x0000000100308c40.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.3/ThreadPoolExecutor.java:1128)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.3/ThreadPoolExecutor.java:628)
at java.lang.Thread.run(java.base@11.0.3/Thread.java:834)
Attachments
Attachments
Issue Links
- relates to
-
SOLR-11960 Add collection level properties
-
- Closed
-