Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-21929

Support `ALTER TABLE table_name ADD COLUMNS(..)` for ORC data source

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.2.0
    • 2.3.0
    • SQL
    • None

    Description

      SPARK-19261 implemented `ADD COLUMNS` at Spark 2.2, but ORC data source is not supported due to its limit.

      scala> sql("CREATE TABLE tab (c1 int, c2 int, c3 int) USING ORC PARTITIONED BY (c3)")
scala> sql("ALTER TABLE tab ADD COLUMNS (c4 int)")
org.apache.spark.sql.AnalysisException:
ALTER ADD COLUMNS does not support datasource table with type ORC.
You must drop and re-create the table for adding the new columns. Tables: `tab`;

      Attachments

        Issue Links

          blocks

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-20901 Feature parity for ORC with Parquet

          • Major - Major loss of function.
          • Open
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-19261 Support `ALTER TABLE table_name ADD COLUMNS(..)` statement

          • Major - Major loss of function.
          • Resolved
          links to

          Pull request #19545 [Github] Pull Request #19545 (dongjoon-hyun)

          Activity

            People

              dongjoon Dongjoon Hyun
              dongjoon Dongjoon Hyun
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: