Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-5463 Improve Parquet support (reliability, performance, and error messages)
  3. SPARK-8580

Test Parquet interoperability and compatibility with other libraries/systems

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.5.0
    • 1.5.0
    • SQL
    • None

    Description

      As we are implementing Parquet backwards-compatibility rules for Spark 1.5.0 to improve interoperability with other systems (reading non-standard Parquet files they generate, and generating standard Parquet files), it would be good to have a set of standard test Parquet files generated by various systems/tools (parquet-thrift, parquet-avro, parquet-hive, Impala, and old versions of Spark SQL) to ensure compatibility.

      Attachments

        Activity

          People

            lian cheng Cheng Lian
            lian cheng Cheng Lian
            Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: