Uploaded image for project: 'Parquet'
  1. Parquet
  2. PARQUET-2238

Spec and parquet-mr disagree on DELTA_BYTE_ARRAY encoding

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Duplicate
    • None
    • None
    • parquet-format, parquet-mr
    • None

    Description

      The spec in parquet-format specifies that DELTA_BYTE_ARRAY is only supported for the physical type BYTE_ARRAY. Yet, parquet-mr also uses it to encode FIXED_LEN_BYTE_ARRAY.

      So, I guess the spec should be updated to include FIXED_LEN_BYTE_ARRAY in the supported types of DELTA_BYTE_ARRAY encoding, or the code should be changed to no longer write this encoding for FIXED_LEN_BYTE_ARRAY.

      I guess changing the spec is more prudent, given that 
      a) the encoding can make sense for FIXED_LEN_BYTE_ARRAY
      and
      b) there might already be countless files written with this encoding / type combination.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              jfinis Jan Finis
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: