Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-17477

IncrementalBlockReport race condition additional edge cases

    XMLWordPrintableJSON

Details

    Description

      HDFS-17453 fixes a race condition between IncrementalBlockReports (IBR) and the Edit Log Tailer which can cause the Standby NameNode (SNN) to incorrectly mark blocks as corrupt when it transitions to Active. There are a few edge cases that HDFS-17453 does not cover.

      For Example:
      1. SNN1 loads the edits for b1gs1 and b1gs2.
      2. DN1 reports b1gs1 to SNN1, so it gets queued for later processing.
      3. DN1 reports b1gs2 to SNN1 so it gets added to the blocks map.
      4. SNN1 transitions to Active (ANN1).
      5. ANN1 processes the pending DN message queue and marks DN1->b1gs1 as corrupt because it was still in the queue.

      Attachments

        Issue Links

          Activity

            People

              dannytbecker Danny Becker
              dannytbecker Danny Becker
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated: