Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13786 Add S3A committers for zero-rename commits to S3 endpoints
  3. HADOOP-13967

S3ABlockOutputStream to support plugin point for different multipart strategies

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 3.0.0-beta1
    • None
    • fs/s3
    • None

    Description

      For 0-rename commits, we need to delay the final commit of a multipart PUT, instead saving the data needed to build that commit into the s3 bucket.

      This means changes to S3ABlockOutputStream so that it can support different policies on how to do this, "classic" and "delayed commit".

      Having this self contained means we can test it in isolation of anything else.

      I'm ignoring the old output stream...we will switch to fast output whenever a special destination path is encountered

      Attachments

        Issue Links

          Activity

            People

              stevel@apache.org Steve Loughran
              stevel@apache.org Steve Loughran
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: