Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-17566

Über-jira: S3A Hadoop 3.3.2 features

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Done
    • 3.3.1
    • 3.3.2
    • fs/s3
    • None

    Attachments

      Issue Links

        is superceded by

        Improvement - An improvement or enhancement to an existing feature or task. HADOOP-18067 Über-jira: S3A Hadoop 3.3.5 features

        • Major - Major loss of function.
        • Resolved
        supercedes

        New Feature - A new feature of the product, which has yet to be developed. HADOOP-16829 Über-jira: S3A Hadoop 3.3.1 features

        • Major - Major loss of function.
        • Resolved
        1.
        		 Add an Audit plugin point for S3A auditing/context Sub-task Resolved Steve Loughran

        100%
        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 22h 20m
        2.
        		 S3A to treat "SdkClientException: Data read has a different length than the expected" as EOFException Sub-task Resolved Bogdan Stolojan

        100%
        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 1.5h
        3.
        		 Add a MkdirOperation for chained S3 operations during mkdir Sub-task Resolved Steve Loughran

        100%
        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 1h 50m
        4.
        		 Magic committer to downgrade abort in cleanup if list uploads fails with access denied Sub-task Resolved Bogdan Stolojan

        100%
        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 0.5h
        5.
        		 S3Guard import can OOM on large imports Sub-task Resolved Steve Loughran

        100%
        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 1h 50m
        6.
        		 Support S3 Access Points Sub-task Resolved Bogdan Stolojan

        100%
        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 16.5h
        7.
        		 Failure of ITestAssumeRole.testRestrictedCommitActions Sub-task Resolved Steve Loughran  
        8.
        		 Re-enable optimized copyFromLocal implementation in S3AFileSystem Sub-task Resolved Bogdan Stolojan

        100%
        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 9h 50m
        9.
        		 S3A AWS Credential provider loading gets confused with isolated classloaders Sub-task Resolved Steve Loughran  
        10.
        		 S3A mkdirs to indicate which parent path element refers to a file Sub-task Resolved Unassigned  
        11.
        		 transient ITestS3AFileContextStatistics failure -read buffer not filled Sub-task Resolved Unassigned  
        12.
        		 s3a rename failed during copy, "Unable to copy part" + 200 error code Sub-task Resolved Unassigned  
        13.
        		 S3a: Failed to reset the request input stream/make S3A uploadPart() retriable Sub-task Resolved Unassigned  
        14.
        		 S3AFileSystem copyFile to propagate etag/version from getObjectMetadata to copy request Sub-task Resolved Unassigned  
        15.
        		 ITestS3AContractGetFileStatusV1List may have consistency issues Sub-task Resolved Unassigned  
        16.
        		 Distcp is unable to determine region with S3 PrivateLink endpoints Sub-task Resolved Unassigned  
        17.
        		 s3a to set fake directory marker contentType to application/x-directory Sub-task Resolved Steve Loughran  
        18.
        		 S3A copy/rename of large files to be parallelized as a multipart operation Sub-task Resolved Unassigned  
        19.
        		 S3A add histogram metrics types for latency, etc. Sub-task Resolved Sean Mackrory  
        20.
        		 s3guard to provide better diags on ddb init failures Sub-task Resolved Unassigned  
        21.
        		 s3a doesn't consider blobs with trailing / and content-length >0 as directories Sub-task Resolved Unassigned  
        22.
        		 Encrypt S3A data client-side with AWS SDK (S3-CSE) Sub-task Resolved Mehakmeet Singh

        100%
        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 14h 40m
        23.
        		 declare that fs.s3a.ext. is a prefix for arbitrary extensions Sub-task Resolved Unassigned  
        24.
        		 improve setting of max connections in AWS client Sub-task Resolved Unassigned  
        25.
        		 AWS Data read stack trace in S3a putObjectDirect Sub-task Resolved Unassigned  
        26.
        		 increase performance of s3guard import command Sub-task Resolved Unassigned  
        27.
        		 S3A to add option fs.s3a.endpoint.region to set AWS region Sub-task Resolved Mehakmeet Singh

        100%
        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 3h
        28.
        		 Upgrade aws-java-sdk to 1.11.1026 Sub-task Resolved Steve Loughran

        100%
        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 1h
        29.
        		 S3A NetworkBinding has a runtime class dependency on a third-party shaded class Sub-task Resolved Steve Loughran

        100%
        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 2h
        30.
        		 S3AFS creation fails "Unable to find a region via the region provider chain." Sub-task Resolved Steve Loughran

        100%
        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 3h 50m
        31.
        		 bytesRead FS statistic showing twice the correct value in S3A Sub-task Resolved Mehakmeet Singh

        100%
        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 2.5h
        32.
        		 S3A ITestPartialRenamesDeletes.testRenameDirFailsInDelete failure: missing directory marker Sub-task Resolved Steve Loughran  
        33.
        		 ITestS3AContractDistCp failing IllegalStateException: WFOPENSSL0030 Running handshake with buffered unwrapped data Sub-task Resolved Unassigned  
        34.
        		 fs.s3a.acl.default not working after S3A Audit feature added Sub-task Resolved Steve Loughran

        100%
        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 3h
        35.
        		 S3A Tests to skip if S3Guard and S3-CSE are enabled. Sub-task Resolved Mehakmeet Singh

        100%
        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 3h 40m
        36.
        		 S3A CSE: minor tuning Sub-task Resolved Mehakmeet Singh

        100%
        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 3.5h
        37.
        		 CredentialProviderFactory.getProviders() recursion loading JCEKS file from s3a Sub-task Resolved Steve Loughran

        100%
        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 1h 20m
        38.
        		 s3a: set fs.s3a.downgrade.syncable.exceptions = true by default Sub-task Resolved Steve Loughran

        100%
        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 3.5h
        39.
        		 AccessPoint verifyBucketExistsV2 always returns false Sub-task Resolved Bogdan Stolojan

        100%
        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 40m

        Activity

          People

            stevel@apache.org Steve Loughran
            stevel@apache.org Steve Loughran
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 97h
                97h