Details

    • Sub-task
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • 2.6.0, 2.7.0, 3.0.0-alpha3
    • None
    • hdfs-client
    • None
    • hdfs-client, heterogeneous storage
    • Patch, Important

    Description

      in default policy, if a file is ONE_SSD, client will prior read the local disk replica rather than the remote ssd replica.
      but now, the pci-e SSD and 10G ethernet make remote read SSD more faster than the local disk.
      HDFS-9666 give us a patch, but the code is not complete and not updated for a long time.
      this sub-task issue give a complete patch and
      we have tested on three machines [ 32 core cpu, 128G mem , 1000M network, 1.2T HDD, 800G SSD(intel P3600) ].
      with this feather, throughput of hbase table(ONE_SSD) is double of which without this feather

      Attachments

        1. HDFS-11942.0.patch
          8 kB
          Fangyuan Deng
        2. HDFS-11942.1.patch
          8 kB
          Fangyuan Deng
        3. ssd-first-enable.png
          618 kB
          Fangyuan Deng
        4. ssd-first-disable(default).png
          619 kB
          Fangyuan Deng

        Activity

          People

            Unassigned Unassigned
            whisper_deng Fangyuan Deng
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated: