Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3164

Use FileChannel.transferTo() when data is read from DataNode.

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.18.0
    • None
    • None
    • Reviewed
    • Hide
      Changed data node to use FileChannel.tranferTo() to transfer block data.
      Show
      Changed data node to use FileChannel.tranferTo() to transfer block data.

    Description

      HADOOP-2312 talks about using FileChannel's transferTo() and transferFrom() in DataNode.

      At the time DataNode neither used NIO sockets nor wrote large chunks of contiguous block data to socket. Hadoop 0.17 does both when data is seved to clients (and other datanodes). I am planning to try using transferTo() in the trunk. This might reduce DataNode's cpu by another 50% or more.

      Once HADOOP-1702 is committed, we can look into using transferFrom().

      Attachments

        1. HADOOP-3164.patch
          14 kB
          Raghu Angadi
        2. HADOOP-3164.patch
          15 kB
          Raghu Angadi
        3. HADOOP-3164.patch
          15 kB
          Raghu Angadi
        4. HADOOP-3164.patch
          16 kB
          Raghu Angadi
        5. HADOOP-3164.patch
          17 kB
          Raghu Angadi
        6. HADOOP-3164.patch
          20 kB
          Raghu Angadi

        Issue Links

          Activity

            People

              rangadi Raghu Angadi
              rangadi Raghu Angadi
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: