Uploaded image for project: 'Parquet'
  1. Parquet
  2. PARQUET-1124

Add new compression codecs to the Parquet spec

    XMLWordPrintableJSON

Details

    • Task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • format-2.4.0
    • parquet-format
    • None

    Description

      After recent tests, I think we should add Zstd to the spec.

      I'm also proposing we add LZ4 because it is widely available and outperforms snappy. As a successor for fast compression but not necessarily good compression ratios, I think it makes sense to have it.

      Attachments

        Issue Links

          blocks

          Bug - A problem which impairs or prevents the functions of the product. PARQUET-1134 Release Parquet format 2.4.0

          • Major - Major loss of function.
          • Resolved
          is related to

          New Feature - A new feature of the product, which has yet to be developed. PARQUET-970 Add Add Lz4 and Zstd compression codecs

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link Compression comparison, Sept 2017

          Web Link PR #70

          Activity

            People

              rdblue Ryan Blue
              rdblue Ryan Blue
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: