Details
-
Sub-task
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
StoragePolicy may be set on an ancestor path and used by some children files.
This storage policy are used by many operations, but there are two special operations:
- ChooseExcessRedundancies
- BlockReconstructionWork
ChooseExcessRedundancies is used to process some excess redundancy replicas. BR and IBR may need to process some excess redundancy replicas.
It means that :
- BR and IBR needs to hold not only the global BM write lock but also the global FS write lock in phase 2.
- BR and IBR needs to hold not only the write lock of last iNode file but also the full directory tree locks in phase 3.
It will results in a low performance.
In order to fix it, there are some solutions.
Solution 1:
- Using a separate thread to process these excess redundancy blocks periodically, such as the Redundancy monitor.
- This thread can hold the full directory tree locks to process these blocks in phase 3.
Solution 2:(preferred)
- Try to set the storage policy to each iNodeFile, so that BM related operations can only involve the iNodeFile to which the processing block belongs.
- It's solution will not cost more memory.
- The storage policy can be set synchronously to all children iNodeFiles. setStoragePolicy, unsetStoragePolicy and rename require this operation.