Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-17204

EC: Reduce unnecessary log when processing excess redundancy.

    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      This is a follow-up of HDFS-16964. We now avoid stale replicas when dealing with redundancy. This may result in redundant replicas not being in the `nonExcess` set when we enter `BlockManager#chooseExcessRedundancyStriped` (because the datanode where the redundant replicas are located has not send FBR yet, so those replicas are filtered out and not added to the `nonExcess` set). A further result is that no excess storage type is selected and the log "excess types chosen for block..." is printed. When a failover occurs, a large number of datanodes become stale, which causes NameNodes to print a large number of unnecessary logs.
      This issue needs to be fixed, otherwise the performance after failover will be affected.

      Attachments

        Issue Links

          Activity

            People

              zhangshuyan Shuyan Zhang
              zhangshuyan Shuyan Zhang
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: