Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6736

FSNamesystem.appendFile() try to remove block from replication queue, but It may not work

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 1.2.1
    • None
    • namenode
    • None

    Description

      the process appendFile() seems to remove block from replication queue, but It may not work , with the code:

      FSNamesystem.appendFile()
      {

      ...
      // Remove block from replication queue.
      updateNeededReplications(last, 0, 0); // last is the block appending
      ...
      }

      UnderReplicatedBlocks.updateNeededReplications(last, 0, 0)

      { NumberReplicas repl = countNodes(last); int curExpectedReplicas = getReplication(last); neededReplications.update(last, repl.liveReplicas(), repl.decommissionedReplicas(), curExpectedReplicas, 0, 0); }

      UnderReplicatedBlocks.update(Block block, int curReplicas,
      int decommissionedReplicas,
      int curExpectedReplicas,
      int curReplicasDelta, int expectedReplicasDelta)
      {
      //while the curReplicasDelta & expectedReplicasDelta are all be 0,
      //the oldReplicas and oldExpectedReplicas while be the same
      // then the curPri == oldPri

      int oldReplicas = curReplicas-curReplicasDelta;
      int oldExpectedReplicas = curExpectedReplicas-expectedReplicasDelta;
      int curPri = getPriority(block, curReplicas, decommissionedReplicas, curExpectedReplicas);
      int oldPri = getPriority(block, oldReplicas, decommissionedReplicas, oldExpectedReplicas);

      // the if condition may get false , coz oldPri == curPri
      // this will never remove(), and it may call the add()
      if(oldPri != LEVEL && oldPri != curPri)

      { remove(block, oldPri); }

      if(curPri != LEVEL && priorityQueues.get(curPri).add(block))

      { ... //log() }

      }

      Attachments

        Activity

          People

            Unassigned Unassigned
            jonsolaris jonsolaris
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:

              Time Tracking

                Estimated:
                Original Estimate - 12h
                12h
                Remaining:
                Remaining Estimate - 12h
                12h
                Logged:
                Time Spent - Not Specified
                Not Specified