Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14147

Backport of HDFS-13056 to the 2.9 branch: "Expose file-level composite CRCs in HDFS"

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • 2.9.0, 2.9.1, 2.9.2
    • None
    • datanode, distcp, hdfs
    • None

    Description

      HDFS-13056, Expose file-level composite CRCs in HDFS which are comparable across different instances/layouts, is a significant feature for storage agnostic CRC comparisons between HDFS and cloud object stores such as S3 and GCS. With the extensively installed base of Hadoop 2, it should make a lot of sense to have the feature in Hadoop 2.

      The plan is to start with the backporting to 2.9, followed by 2.8 and 2.7 in that order.

      Attachments

        1. HDFS-14147.pdf
          118 kB
          Yan
        2. HDFS-14147-branch-2.9-001.patch
          145 kB
          Yan
        3. HDFS-14147-branch-2.9-001.patch
          145 kB
          Yan

        Issue Links

          is duplicated by

          Task - A task that needs to be done. HDFS-14496 Backport HDFS-13056 into branch-2.8/2.9/2

          • Major - Major loss of function.
          • Resolved
          relates to

          New Feature - A new feature of the product, which has yet to be developed. HDFS-13056 Expose file-level composite CRCs in HDFS which are comparable across different instances/layouts

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #446

          Web Link GitHub Pull Request #446

          Activity

            People

              yzhou2001 Yan
              yzhou2001 Yan
              Votes:
              3 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated: