Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-17566 Über-jira: S3A Hadoop 3.3.2 features
  3. HADOOP-17928

s3a: set fs.s3a.downgrade.syncable.exceptions = true by default

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.3.1
    • 3.3.2
    • fs/s3

    Description

      HADOOP-17597 set policy of reacting to hsync() on an s3 output stream to be one of :Fail, warn, with default == fail.

      I propose downgrading this to warn. We've done it internally, after having it on fail long enough to identify which processes were doing either of

      • having unrealistic expectations about the output stream (fix: move off s3)
      • were using hflush() as a variant of flush(), with the failure being an over-reaction

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            stevel@apache.org Steve Loughran
            stevel@apache.org Steve Loughran
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 3.5h
                3.5h

                Slack

                  Issue deployment