Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      StoragePolicy may be set on an ancestor path and used by some children files.

       

      This storage policy are used by many operations, but there are two special operations:

      • ChooseExcessRedundancies 
      • BlockReconstructionWork

       

      ChooseExcessRedundancies is used to process some excess redundancy replicas. BR and IBR may need to process some excess redundancy replicas.

      It means that :

      • BR and IBR needs to hold not only the global BM write lock but also the global FS write lock in phase 2.
      • BR and IBR needs to hold not only the write lock of last iNode file but also the full directory tree locks in phase 3.

       

      It will results in a low performance. 

       

      In order to fix it, there are some solutions.

      Solution 1: 

      • Using a separate thread to process these excess redundancy blocks periodically, such as the Redundancy monitor.
      • This thread can hold the full directory tree locks to process these blocks in phase 3.

       

      Solution 2:(preferred)

      • Try to set the storage policy to each iNodeFile, so that BM related operations can only involve the iNodeFile to which the processing block belongs.
      • It's solution will not cost more memory.
      • The storage policy can be set synchronously to all children iNodeFiles. setStoragePolicy, unsetStoragePolicy and rename require this operation.

      Attachments

        Activity

          People

            xuzq_zander ZanderXu
            xuzq_zander ZanderXu
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: