Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-9413

Introduce common utils for File#setReadable/Writable/Executable and File#canRead/Write/Execute that work cross-platform

VotersStop watchingWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 2.1.0-beta
    • None
    • None
    • Reviewed

    Description

      So far, we've seen many unittest and product bugs in Hadoop on Windows because Java's APIs that manipulate with permissions do not work as expected. We've addressed many of these problems on one-by-one basis (by either changing code a bit or disabling the test). While debugging the remaining unittest failures we continue to run into the same patterns of problems, and instead of addressing them one-by-one, I propose that we expose a set of equivalent wrapper APIs that will work well for all platforms.

      Scanning thru the codebase, this will actually be a simple change as there are very few places that use File#setReadable/Writable/Executable and File#canRead/Write/Execute (5 files in Common, 9 files in HDFS).

      HADOOP-8973 contains additional context on the problem.

      Attachments

        1. HADOOP-9413.commonfileutils.5.patch
          25 kB
          Ivan Mitic
        2. HADOOP-9413.commonfileutils.4.patch
          25 kB
          Ivan Mitic
        3. HADOOP-9413.commonfileutils.3.patch
          22 kB
          Ivan Mitic
        4. HADOOP-9413.commonfileutils.2.patch
          22 kB
          Ivan Mitic
        5. HADOOP-9413.commonfileutils.patch
          17 kB
          Ivan Mitic

        Issue Links

        is depended upon by

        Bug - A problem which impairs or prevents the functions of the product. HDFS-4610 Move to using common utils FileUtil#setReadable/Writable/Executable and FileUtil#canRead/Write/Execute

        • Major - Major loss of function.
        • Closed

        Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-5177 Move to common utils FileUtil#setReadable/Writable/Executable and FileUtil#canRead/Write/Execute

        • Major - Major loss of function.
        • Closed

        Bug - A problem which impairs or prevents the functions of the product. YARN-506 Move to common utils FileUtil#setReadable/Writable/Executable and FileUtil#canRead/Write/Execute

        • Major - Major loss of function.
        • Closed
        is related to

        New Feature - A new feature of the product, which has yet to be developed. HADOOP-8562 Enhancements to support Hadoop on Windows Server and Windows Azure environments

        • Major - Major loss of function.
        • Closed
        relates to

        Bug - A problem which impairs or prevents the functions of the product. HADOOP-9551 Backport common utils introduced with HADOOP-9413 to branch-1-win

        • Major - Major loss of function.
        • Resolved

        Bug - A problem which impairs or prevents the functions of the product. HADOOP-8973 DiskChecker cannot reliably detect an inaccessible disk on Windows with NTFS ACLs

        • Major - Major loss of function.
        • Closed

        Improvement - An improvement or enhancement to an existing feature or task. HADOOP-9590 Move to JDK7 improved APIs for file operations when available

        • Major - Major loss of function.
        • Open

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ivanmi Ivan Mitic
            ivanmi Ivan Mitic
            Votes:
            0 Vote for this issue
            Watchers:
            7 Stop watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment