Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-17458

Remove unnecessary BP lock in ReplicaMap

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.4.0
    • None
    • datanode

    Description

      In HDFS-16429 we make LightWeightResizableGSet to be thread safe, and in HDFS-16511  we change some methods in ReplicaMap to acquire read lock instead of acquiring write lock.

      This PR try to remove unnecessary Block_Pool read lock further.

      Recently, I performed stress tests on datanodes to measure their read/write operations/second.

      Before we removing some lock,  it can only achieve ~2K write ops. After optimizing, it can achieve more than 5K write ops.

      Attachments

        Issue Links

          Activity

            People

              zhanghaobo farmmamba
              zhanghaobo farmmamba
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: