Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4996

ClientProtocol#metaSave can be made idempotent by overwriting the output file instead of appending to it

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 2.1.0-beta, 3.0.0-alpha1
    • 2.1.0-beta
    • namenode
    • None
    • Reviewed
    • The dfsadmin -metasave command has been changed to overwrite the output file. Previously, this command would append to the output file if it already existed.

    Description

      ClientProtocol#metaSave opens its output file for append. This prevents the operation from being idempotent, because retries can cause unpredictable side effects (single copy of the output vs. multiple copies). As discussed on HDFS-4974, the operation could be made idempotent by changing this to open the file for overwrite instead of append.

      Attachments

        1. HDFS-4996.1.patch
          6 kB
          Chris Nauroth
        2. HDFS-4996.2.patch
          8 kB
          Chris Nauroth

        Issue Links

          Activity

            People

              cnauroth Chris Nauroth
              cnauroth Chris Nauroth
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: