[HADOOP-18752] Change fs.s3a.directory.marker.retention to "keep" - ASF JIRA

Voters

Watch issue

Watchers

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Target Version/s:

3.4.0, 3.3.9
Hadoop Flags:

Incompatible change
Release Note:

Hide
The s3a connector no longer deletes directory markers by default, which speeds up write operations, reduces iO throttling and saves money. this can cause problems with older hadoop releases trying to write to the same bucket. (Hadoop 3.3.0; Hadoop 3.2.x before 3.2.2, and all previous releases). Set "fs.s3a.directory.marker.retention" to "delete" for backwards compatibility

Show
The s3a connector no longer deletes directory markers by default, which speeds up write operations, reduces iO throttling and saves money. this can cause problems with older hadoop releases trying to write to the same bucket. (Hadoop 3.3.0; Hadoop 3.2.x before 3.2.2, and all previous releases). Set "fs.s3a.directory.marker.retention" to "delete" for backwards compatibility