Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-8450

Add support for zstd in parquet

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • Impala 3.3.0
    • Backend

    Description

      PARQUET-970 added these codecs to the format. We have LZ4 in the toolchain already and I just added zstd: https://gerrit.cloudera.org/#/c/13079/

      These codec probably offer a better trade-off of density and speed than snappy or gzip.

      https://github.com/apache/arrow/pull/807/files might be a useful crib sheet for how to add a compressor.

      LZ4 support will be added using: https://issues.apache.org/jira/browse/IMPALA-8617

      Attachments

        Activity

          People

            arawat Abhishek Rawat
            tarmstrong Tim Armstrong
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: