Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14988

HDFS should avoid read/write data from slow disks.

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.1.1, 3.2.1
    • None
    • block placement, datanode
    • None

    Description

      Slow disk causes real-time service(such as HBase ) become slowdown.

      The slow disk detection is added to the HDFS-11461, but only the slow disk is recorded in the Metric. I hope to further handle the detected slow disk. In my view, the slow disk can be added to the read data policy. If the block is on the slow disk of the DataNode, the block of other DataNodes is selected. For write data, slow disks can be added to the data write policy. We can remove the slow disk from all disks and then select a disk to write data based on dfs.datanode.fsdataset.volume.choosing.policy.

      Attachments

        Activity

          People

            Unassigned Unassigned
            jj336013 yimeng
            Votes:
            1 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated: