Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4526

If persistBlocks enabled in Namenode, hflush call can avoid persisting blocks with fsync

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Not A Problem
    • 2.0.3-alpha, 3.0.0-alpha1
    • None
    • namenode
    • None

    Description

      If persists blocks enabled/in HA, evenry getAdditionalBlock will persist the blocks.
      So, if user calls Hflush, then client can simply ensure data flushed to DNs and need not worry about fsync.
      Since we can not depend upon client side configuration about persistBlocks, we can just make a fsync call to NN and check whether persistBlocks enabled. If it is enabled, then simply return as it would have already persisted. So, that below peice of code execution can be avoided for every blocks and hflush calls

      from fsync:

       writeLock();
          try {
            checkOperation(OperationCategory.WRITE);
            if (isInSafeMode()) {
              throw new SafeModeException("Cannot fsync file " + src, safeMode);
            }
            INodeFileUnderConstruction pendingFile  = checkLease(src, clientName);
            dir.persistBlocks(src, pendingFile);
          } finally {
            writeUnlock();
          }
          getEditLog().logSync();
      

      Attachments

        1. HDFS-4526.patch
          0.7 kB
          Uma Maheswara Rao G

        Activity

          People

            umamaheswararao Uma Maheswara Rao G
            umamaheswararao Uma Maheswara Rao G
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: