Bulk Operation

HADOOP-16829 Add option to downgrade S3A rejection of Syncable to warning

HADOOP-16829 typo in MagicCommitTracker

HADOOP-16829 magic committer to be enabled for all S3 buckets

HADOOP-16829 S3A docs to state s3 is consistent, deprecate S3Guard

HADOOP-16829 ITestAssumeRole.testAssumeRoleBadInnerAuth failure

HADOOP-16829 Magic committer files don't have the count of bytes written collected by spark

HADOOP-16829 enable s3a magic committer by default

HADOOP-16829 AbstractS3ATokenIdentifier to issue date in UTC

HADOOP-16829 ITestS3ADeleteCost.testDirMarkersFileCreation failure

HADOOP-16829 AbstractS3ATokenIdentifier to set issue date == now

HADOOP-16829 ITestS3AContractRename failing against stricter tests

HADOOP-16829 Upgrade aws-java-sdk to 1.11.901

HADOOP-16829 Intermittent S3AInputStream failures: Premature end of Content-Length delimited message body etc

HADOOP-16829 S3A marker tool mixes up -min and -max

HADOOP-16829 S3A committer to support concurrent jobs with same app attempt ID & dest dir

HADOOP-16829 FileSystem.get to support slow-to-instantiate FS clients

HADOOP-16829 S3AInputStream to be resilient to faiures in abort(); translate AWS Exceptions

HADOOP-16829 ITestCustomSigner fails with gcs s3 compatible endpoint.

HADOOP-16829 S3A to always probe S3 in S3A getFileStatus on non-auth paths

HADOOP-16829 s3a rename() now requires s3:deleteObjectVersion permission

HADOOP-16829 HADOOP-17244. S3A directory delete tombstones dir markers prematurely.

HADOOP-16829 Backport HADOOP-13230 listing changes for preserved directory markers to 3.1.x

HADOOP-16829 improve s3guard markers command line tool

HADOOP-16829 Renaming a file under a sibling empty directory doesn't delete dest dir's marker

HADOOP-16829 Backport HADOOP-13230 list/getFileStatus changes for preserved directory markers

HADOOP-16829 Decrease size of s3a dependencies

HADOOP-16829 add way for s3a to recognise buckets with "." in name and switch to path access

HADOOP-16829 Handle transient stream read failures in FileSystem contract tests

HADOOP-16829 ITestS3AEncryptionWithDefaultS3Settings fails if default bucket encryption != KMS

HADOOP-16829 ITestS3GuardOutOfBandOperations testListingDelete[auth=false] fails on unversioned bucket

HADOOP-16829 whitespace not allowed in paths when saving files to s3a via committer

HADOOP-16829 S3A staging committer committing duplicate files

HADOOP-16829 S3A to support additional token issuers

HADOOP-16829 ITestS3AConfiguration proxy tests fail when bucket probes == 0

HADOOP-16829 s3a to not need wildfly on the classpath

HADOOP-16829 HADOOP-16953. tune s3guard disabled warnings

HADOOP-16829 ITestS3GuardOutOfBandOperations.testListingDelete failing on versioned bucket

HADOOP-16829 s3guard can't init table if caller doesn't have tag permissions

HADOOP-16829 mkdir on s3a should not be sensitive to trailing '/'

HADOOP-16829 Add some Abortable.abort() interface for streams etc which can be terminated

HADOOP-16829 s3a mkdirs() to not check dest for a dir marker

HADOOP-16829 S3A FS deleteOnExit to skip the exists check

HADOOP-16829 Report on S3A cached 404 recovery better

HADOOP-16829 add experimental optimization of s3a directory marker handling

HADOOP-16829 catch and downgrade all exceptions trying to load openssl native libs through wildfly

HADOOP-16829 s3a mkdir path/ can add 404 to S3 load balancers

HADOOP-16829 job commit failure in S3A MR magic committer test

HADOOP-16829 distcp -update to S3A; abfs, etc always overwrites due to block size mismatch

HADOOP-16829 Possible NPE in S3A MultiObjectDeleteSupport error handling

HADOOP-16829 Improve S3A rename resilience

HADOOP-16829 s3guard LimitExceededException -too many tables

HADOOP-16829 Retrive modtime of PUT file from store, via response or HEAD

HADOOP-16829 intermittent failure of ITestCommitOperations: too many s3guard writes

HADOOP-16829 ITestS3ARemoteFileChanged tests fail if you set the bucket to versionid tracking

HADOOP-16829 IAM role created by S3A DT doesn't include DynamoDB scan

HADOOP-16829 S3A FullCredentialsTokenBinding fails if local credentials are unset

HADOOP-16829 make sure staging committers collect DTs for the staging FS

HADOOP-16829 S3AFilesystem.initiateRename() can skip check on dest.parent status if src has same parent

HADOOP-16829 S3A doesn't actually verify paths have the correct authority

HADOOP-16829 Intermittent failure of ITestS3GuardConcurrentOps#testConcurrentTableCreations

HADOOP-16829 S3Guard prune to only remove auth dir marker if files (not tombstones) are removed

HADOOP-16829 S3AFileSystem.getContentSummary() to use listFiles(recursive)

HADOOP-16829 Intermittent failure of ITestS3GuardToolDynamoDB#testDynamoDBInitDestroyCycle

HADOOP-16829 Fix ITestS3GuardToolLocal#testInitNegativeRead test failure

HADOOP-16829 S3a getFileStatus to update DDB if an S3 query returns etag/versionID

HADOOP-16829 S3Guard init command uses global settings, not those of target bucket

HADOOP-16829 Clock skew can cause S3Guard to think object metadata is out of date

HADOOP-16829 ITestS3GuardToolDynamoDB.testDynamoDBInitDestroyCycle fails if test bucket isn't on demand

HADOOP-16829 ZookeeperMetadataStore: Use Zookeeper as S3Guard backend store

HADOOP-16829 Add option for a prefix to put in front of every s3guard table

HADOOP-16829 add extra S3A MPU test to see what happens if a file is created during the MPU

HADOOP-16829 S3Guard: Optimize performance of handling OOB operations in non-authoritative mode

HADOOP-16829 Add some tests about S3 timestamp tracking

HADOOP-16829 S3A statistic collection underrecords bytes written in helper threads

HADOOP-16829 S3A Client to add explicit support for versioned stores

HADOOP-16829 getFileChecksum() needs to adopt S3Guard

HADOOP-16829 tag S3GuardTool entry points as limitedPrivate("management-tools")/evolving

HADOOP-16829 Improve throttling on S3Guard DDB batch retries

HADOOP-16829 S3guard: add inconsistency detection metrics

HADOOP-16829 intermittent failure of ITestS3GuardListConsistency.testInconsistentS3ClientDeletes in parallel runs

HADOOP-16829 S3Guard to self update on directory listings of S3

HADOOP-16829 reduce/tune read failure fault injection on inconsistent client

HADOOP-16829 S3A Input Stream bytes read counter isn't getting through to StorageStatistics/instrumentation properly

HADOOP-16829 Scheme assertion in S3Guard DynamoDBMetadataStore::checkPath is unnecessarily restrictive

HADOOP-16829 S3guard mistakes root URI without / as non-absolute path

HADOOP-16829 S3Guard instrumentation to include cost of DynamoDB ops as metric

HADOOP-16829 initial part uploads seem to block unnecessarily in S3ABlockOutputStream

HADOOP-16829 S3A authenticators to log origin of .secret.key options

HADOOP-16829 Ensure controls in-place to prevent clients with significant clock skews pruning aggressively

HADOOP-16829 s3a test can hang in teardown with network problems

HADOOP-16829 Use error code detail in AWS server responses for finer grained exceptions

HADOOP-16829 Add more s3guard metrics

HADOOP-16829 s3a: auto-detect region for bucket and use right endpoint

HADOOP-16829 Improve DynamoDB schema update story

HADOOP-16829 S3A Support Shared Instance Profile Credentials Across All Hadoop Nodes

HADOOP-16829 Add some Java-8 friendly way to work with RemoteIterable, especially listings

HADOOP-16829 Possible for modified configuration to leak into metadatastore in S3GuardTool

HADOOP-16829 S3AFileSystem silently deletes "fake" directories when writing a file.

HADOOP-16829 improvements to S3GuardTool destroy command

HADOOP-16829 add tests/docs for HAR files on s3a

HADOOP-16829 S3guard metadata stores to support millions of entries

HADOOP-16829 log DNS addresses on s3a init

HADOOP-16829 s3a to instrument duration of HTTP calls

HADOOP-16829 Collect AwsSdkMetrics in S3A FileSystem IOStatistics

HADOOP-16829 Optimize uses of FS operations in the ASF analysis frameworks and libraries

HADOOP-16829 export s3a BlockingThreadPoolExecutorService pool info (size, load) as gauges

HADOOP-16829 Optimize getFileStatus in S3A

HADOOP-16829 s3a directory housekeeping operations to be done in async thread

HADOOP-16829 AmazonClient 30x exceptions to include redirect URL in message

HADOOP-16829 S3A to optionally retain directory markers

HADOOP-16829 s3a create() doesn't check for an ancestor path being a file