Details
Description
In HDFS, when removing some file, NN will also remove all its blocks from BlocksMap, and send DNA_INVALIDATE (invalidate blocks) commands to datanodes. After datanodes successfully delete the block replicas, will report DELETED_BLOCK to NameNode.
snippet code logic in BlockManager#processIncrementalBlockReport is as following
case DELETED_BLOCK:
removeStoredBlock(storageInfo, getStoredBlock(rdbi.getBlock()), node);
...
private void removeStoredBlock(DatanodeStorageInfo storageInfo, Block block, DatanodeDescriptor node) { if (shouldPostponeBlocksFromFuture && namesystem.isGenStampInFuture(block)) { queueReportedBlock(storageInfo, block, null, QUEUE_REASON_FUTURE_GENSTAMP); return; } removeStoredBlock(getStoredBlock(block), node); }
In EC branch, we add getStoredBlock. There is NullPointerException when handling DELETED_BLOCK of incrementalBlockReport from DataNode after delete a file, since the block is already removed, we need to check.