Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-16593

Correct inaccurate BlocksRemoved metric on DataNode side

    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      When tracing the root cause of production issue, I found that the BlocksRemoved metric on Datanode size was inaccurate.

      case DatanodeProtocol.DNA_INVALIDATE:
            //
            // Some local block(s) are obsolete and can be 
            // safely garbage-collected.
            //
            Block toDelete[] = bcmd.getBlocks();
            try {
              // using global fsdataset
              dn.getFSDataset().invalidate(bcmd.getBlockPoolId(), toDelete);
            } catch(IOException e) {
              // Exceptions caught here are not expected to be disk-related.
              throw e;
            }
            dn.metrics.incrBlocksRemoved(toDelete.length);
            break;
      

      Because even if the invalidate method throws an exception, some blocks may have been successfully deleted internally.

      Attachments

        Issue Links

          Activity

            People

              xuzq_zander ZanderXu
              xuzq_zander ZanderXu
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 50m
                  50m