Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-18477 Über-jira: S3A Hadoop 3.3.9 features
  3. HADOOP-13611

FileSystem/s3a processDeleteOnExit to skip the exists() check

Add voteVotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Trivial
    • Resolution: Unresolved
    • 2.7.3
    • None
    • fs, fs/s3
    • None

    Description

      If you look at FileSystem.processDeleteOnExit(), it does an exists() check for each entry, before calling delete().

      That exists() check is superfluous; on s3 it add an extra 1-4 HTTP GETs

      This could be fixed with a subclass in s3a to avoid it, but as the call is superfluous in all filesystems, it could be removed in FileSystem and so picked up by all object stores.

      Attachments

        Issue Links

        is duplicated by

        Sub-task - The sub-task of the issue HADOOP-16877 S3A FS deleteOnExit to skip the exists check

        • Major - Major loss of function.
        • Resolved
        is related to

        Bug - A problem which impairs or prevents the functions of the product. HADOOP-18340 deleteOnExit does not work with S3AFileSystem

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Resolved
        links to

        Web Link #1924

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            stevel@apache.org Steve Loughran
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:

              Slack

                Issue deployment