Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-9761

Inconsistent metadata handling with ALTER TABLE

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Incomplete
    • 1.4.1
    • None
    • SQL
    • Ubuntu on AWS

    Description

      Schema changes made with ALTER TABLE are not shown in DESCRIBE TABLE. The table in question was created with HiveContext.read.json().

      Steps:

      1. alter table dimension_components add columns (z string); succeeds.
      2. describe dimension_components; does not show the new column, even after restarting spark-sql.
      3. A second alter table dimension_components add columns (z string); fails with RROR exec.DDLTask: org.apache.hadoop.hive.ql.metadata.HiveException: Duplicate column name: z

      Full spark-sql output here.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              simeons Simeon Simeonov
              Votes:
              5 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: