Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7285 Erasure Coding Support inside HDFS
  3. HDFS-8428

Erasure Coding: Fix the NullPointerException when deleting file

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • HDFS-7285
    • None
    • None
    • Reviewed

    Description

      In HDFS, when removing some file, NN will also remove all its blocks from BlocksMap, and send DNA_INVALIDATE (invalidate blocks) commands to datanodes. After datanodes successfully delete the block replicas, will report DELETED_BLOCK to NameNode.

      snippet code logic in BlockManager#processIncrementalBlockReport is as following

      case DELETED_BLOCK:
              removeStoredBlock(storageInfo, getStoredBlock(rdbi.getBlock()), node);
              ...
      
      private void removeStoredBlock(DatanodeStorageInfo storageInfo, Block block,
            DatanodeDescriptor node) {
          if (shouldPostponeBlocksFromFuture &&
              namesystem.isGenStampInFuture(block)) {
            queueReportedBlock(storageInfo, block, null,
                QUEUE_REASON_FUTURE_GENSTAMP);
            return;
          }
          removeStoredBlock(getStoredBlock(block), node);
        }
      

      In EC branch, we add getStoredBlock. There is NullPointerException when handling DELETED_BLOCK of incrementalBlockReport from DataNode after delete a file, since the block is already removed, we need to check.

      Attachments

        Activity

          People

            hitliuyi Yi Liu
            hitliuyi Yi Liu
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: