Details
-
Bug
-
Status: Closed
-
Blocker
-
Resolution: Fixed
-
None
-
3
Description
#6847 extends the `InProcessLockProvider` to support multiple tables in the same process, by having an in-memory static final map storing the mapping of the table base path to the read-write reentrant lock, so that the writer uses the corresponding lock based on the base path. When closing the lock provider, `close()` removes the lock entry. Since `close()` is called when closing the write client, the lock is removed and subsequent concurrent writers will get a different lock instance on the same table, causing the locking mechanism on the same table to be useless. Take the following example where three writers write to the same table concurrently and need to acquire the in-process lock:
```
Writer 1: lock |----------------| unlock and close
Writer 2: try lock | ... lock |------| unlock and close
Writer 3: try lock | ... lock |------| unlock and close
```
after Writer 1 releases the lock and closes the lock provider, the lock instance is removed from the map, and Writer 3 will get a different lock instance compared to Writer 2.
Attachments
Issue Links
- fixes
-
HUDI-6176 Fix flaky test testArchivalWithMultiWriters
- Closed
- links to