[HADOOP-16499] S3A retry policy to be exponential - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Sub-task
Status: Resolved
Priority: Critical
Resolution: Fixed
Affects Version/s: 3.2.0, 3.1.2
Fix Version/s: 3.3.0
Component/s: fs/s3
Labels:
None

Target Version/s:

3.3.0
Release Note:
The S3A filesystem now backs off exponentially on failures. if you have customized the fs.s3a.retry.limit and fs.s3a.retry.interval options you may wish to review these settings

Description

the fixed s3a retry policy doesnt leave big enough gaps for cached 404s to expire; we cant recover from this

~~HADOOP-16490~~ is a full fix for this, but one we can backport is moving from fixed to exponential retries

Attachments

Issue Links

is related to

HADOOP-16490 Avoid/handle cached 404s during S3A file creation

Resolved

links to

GitHub Pull Request #1246

Activity

People

Assignee:: Steve Loughran

Reporter:: Steve Loughran

Votes:: 0 Vote for this issue

Watchers:: 4 Start watching this issue

Dates

Created:: 07/Aug/19 15:42

Updated:: 20/Aug/19 16:57

Resolved:: 09/Aug/19 15:32