Uploaded image for project: 'Parquet'
  1. Parquet
  2. PARQUET-2158

Upgrade Hadoop dependency to version 3.2.0

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.13.0
    • 1.13.0
    • parquet-mr
    • None

    Description

      Parquet still builds against Hadoop 2.10. This is very out of date and does not work with java 11, let alone later releases.

      Upgrading the dependency to Hadoop 3.2.0 makes the release compatible with java 11, and lines up with active work on HADOOP-18287, Provide a shim library for modern FS APIs

      This will significantly speed up access to columnar data, especially in cloud stores.

      Attachments

        Issue Links

          relates to

          New Feature - A new feature of the product, which has yet to be developed. HADOOP-18287 Provide a shim library for modern FS APIs

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. PARQUET-2165 remove deprecated PathGlobPattern and DeprecatedFieldProjectionFilter to compile on hadoop 3.2+

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-12436 GlobPattern regex library has performance issues with wildcard characters

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              Unassigned Unassigned
              stevel@apache.org Steve Loughran
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: