Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12666

Support Microsoft Azure Data Lake - as a file system in Hadoop

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.0.0-alpha1
    • fs/adl, tools
    • None
    • Reviewed
    • Hadoop now supports integration with Azure Data Lake as an alternative Hadoop-compatible file system. Please refer to the Hadoop site documentation of Azure Data Lake for details on usage and configuration.

    Description

      Description

      This JIRA describes a new file system implementation for accessing Microsoft Azure Data Lake Store (ADL) from within Hadoop. This would enable existing Hadoop applications such has MR, HIVE, Hbase etc.., to use ADL store as input or output.

      ADL is ultra-high capacity, Optimized for massive throughput with rich management and security features. More details available at https://azure.microsoft.com/en-us/services/data-lake-store/

      Attachments

        1. Create_Read_Hadoop_Adl_Store_Semantics.pdf
          431 kB
          Vishwajeet Dusane
        2. HADOOP-12666-002.patch
          218 kB
          Vishwajeet Dusane
        3. HADOOP-12666-003.patch
          220 kB
          Vishwajeet Dusane
        4. HADOOP-12666-004.patch
          220 kB
          Vishwajeet Dusane
        5. HADOOP-12666-005.patch
          220 kB
          Vishwajeet Dusane
        6. HADOOP-12666-006.patch
          233 kB
          Vishwajeet Dusane
        7. HADOOP-12666-007.patch
          107 kB
          Vishwajeet Dusane
        8. HADOOP-12666-008.patch
          108 kB
          Vishwajeet Dusane
        9. HADOOP-12666-009.patch
          108 kB
          Vishwajeet Dusane
        10. HADOOP-12666-010.patch
          106 kB
          Vishwajeet Dusane
        11. HADOOP-12666-011.patch
          104 kB
          Vishwajeet Dusane
        12. HADOOP-12666-012.patch
          110 kB
          Christopher Douglas
        13. HADOOP-12666-013.patch
          110 kB
          Chris Nauroth
        14. HADOOP-12666-014.patch
          111 kB
          Christopher Douglas
        15. HADOOP-12666-015.patch
          111 kB
          Vishwajeet Dusane
        16. HADOOP-12666-016.patch
          112 kB
          Vishwajeet Dusane
        17. HADOOP-12666-1.patch
          179 kB
          Vishwajeet Dusane

        Issue Links

        Dependent

        Improvement - An improvement or enhancement to an existing feature or task. HADOOP-13037 Refactor Azure Data Lake Store as an independent FileSystem

        • Major - Major loss of function.
        • Resolved

        Test - A new unit, integration or system test. HADOOP-12875 [Azure Data Lake] Support for contract test and unit test cases

        • Major - Major loss of function.
        • Resolved
        is blocked by

        Wish - General wishlist item. HDFS-9938 Support extensions to WebHdfsFileSystem

        • Major - Major loss of function.
        • Resolved
        is depended upon by

        Bug - A problem which impairs or prevents the functions of the product. HADOOP-13242 Authenticate to Azure Data Lake using client ID and keys

        • Major - Major loss of function.
        • Resolved

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            vishwajeet.dusane Vishwajeet Dusane
            vishwajeet.dusane Vishwajeet Dusane
            Votes:
            3 Vote for this issue
            Watchers:
            33 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - 336h
                336h
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 336h
                336h

                Slack

                  Issue deployment