Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15620 Über-jira: S3A phase VI: Hadoop 3.3 features
  3. HADOOP-16379

S3AInputStream#unbuffer should merge input stream stats into fs-wide stats

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.2.0
    • 3.3.0
    • fs/s3
    • None

    Description

      HADOOP-14747 added support for CanUnbuffer to S3A. One issue for applications that rely on the CanUnbuffer functionality, is that the current S3A code only merges input stream stats when the stream is closed. Applications using CanUnbuffer might not close streams for a long time, so any stats reported by input streams don't get reported in the filesystem-wide metrics.

      This JIRA proposes merging input stream statistics when S3AInputStream#unbuffer is called.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            stakiar Sahil Takiar
            stakiar Sahil Takiar
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment