Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7285 Erasure Coding Support inside HDFS
  3. HDFS-8145

Fix the editlog corruption exposed by failed TestAddStripedBlocks

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • HDFS-7285
    • None
    • None
    • Reviewed

    Description

      TestAddStripedBlocks failed with some editlog corruption. Did some debugging, I can see at least two issues:

      1. DFSStripedOutputStream tries to send out an empty packet to close the block even if writing 0 bytes
      2. Because of the above, NN tries to close the file. This exposes another bug in BlockInfoStriped, which writes its data/parity block numbers into the close editlog but do not read them while loading.

      This jira aims to only fix #2 and HDFS-8166 is used for fixing #1.

      Attachments

        1. HDFS-8145.000.patch
          32 kB
          Jing Zhao
        2. HDFS-8145.001.patch
          14 kB
          Jing Zhao
        3. HDFS-8145.002.patch
          14 kB
          Jing Zhao

        Activity

          People

            jingzhao Jing Zhao
            jingzhao Jing Zhao
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: