Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7285 Erasure Coding Support inside HDFS
  3. HDFS-8453

Erasure coding: properly handle start offset for internal blocks in a block group

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • HDFS-7285
    • None
    • None
    • Reviewed

    Description

        void actualGetFromOneDataNode(final DNAddrPair datanode,
          ...
            LocatedBlock block = getBlockAt(blockStartOffset);
          ...
            fetchBlockAt(block.getStartOffset());
      

      The blockStartOffset here is from inner block. For parity blocks, the offset will overlap with the next block group, and we may end up with fetching wrong block. So we have to assign a meaningful start offset for internal blocks in a block group, especially for parity blocks.

      Attachments

        1. HDFS-8453-HDFS-7285.00.patch
          17 kB
          Zhe Zhang
        2. HDFS-8453-HDFS-7285.01.patch
          20 kB
          Zhe Zhang

        Issue Links

          Activity

            People

              zhz Zhe Zhang
              zhz Zhe Zhang
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: