Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-29376

Upgrade Apache Arrow to 0.15.1

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0
    • 3.0.0
    • PySpark, SQL
    • None

    Description

      Apache Arrow 0.15.0 was just released see https://arrow.apache.org/blog/2019/10/06/0.15.0-release/

      There are a number of fixes and improvements including a change to the binary IPC format https://issues.apache.org/jira/browse/ARROW-6313.

      The next planned release will be 1.0.0, so it would be good to upgrade Spark as a preliminary step.

      Updated to use Apache Arrow 0.15.1, which was released soon after 0.15.0 to include some important bug fixes.

      change log at https://arrow.apache.org/release/0.15.1.html

      Attachments

        Issue Links

          is depended upon by

          Sub-task - The sub-task of the issue SPARK-29493 Add MapType support for Arrow Java

          • Major - Major loss of function.
          • Resolved
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-26566 Upgrade apache/arrow to 0.12.0

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-29378 Upgrade SparkR to use Arrow 0.15 API

          • Major - Major loss of function.
          • Resolved
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-32312 Upgrade Apache Arrow to 1.0.0

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #26133

          Activity

            People

              Unassigned Unassigned
              bryanc Bryan Cutler
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: