Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12943 Consistent Reads from Standby Node
  3. HDFS-13122

Tailing edits should not update quota counts on ObserverNode

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • None
    • None
    • hdfs, namenode
    • None

    Description

      Currently in FSImage#loadEdits(), after applying a set of edits, we call

      updateCountForQuota(target.getBlockManager().getStoragePolicySuite(), target.dir.rootDir);
      

      to update the quota counts for the entire namespace, which can be very expensive. This makes sense if we are about to become the ANN, since we need valid quotas, but not on an ObserverNode which does not need to enforce quotas.

      This is related to increasing the frequency with which the SbNN can tail edits from the ANN to decrease the lag time for transactions to appear on the Observer.

      Attachments

        Issue Links

          Activity

            People

              xkrogen Erik Krogen
              xkrogen Erik Krogen
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: