Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-6056

The AvroSerDe gives out BadSchemaException if a partition is added to the table

Log workAgile BoardRank to TopRank to BottomBulk Copy AttachmentsBulk Move AttachmentsAdd voteVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • 0.11.0
    • None
    • Database/Schema
    • None

    • amazon EMR (hadoop Amazon 1.0.3), avro-1.7.5

    • avro, hive

    Description

      While creating an external table if I do not add a partition, I am able to read files using following format: 

      CREATE external TABLE event
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
LOCATION 's3n://test-event/input/2013/14/10'
TBLPROPERTIES ('avro.schema.literal' = '..some schema..');

      but if I add a partition based on date

      CREATE external TABLE event
PARTITIONED BY (ds STRING)
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
LOCATION 's3n://test-event/input/'
TBLPROPERTIES ('avro.schema.literal' = '..some schema..');

ALTER TABLE event ADD IF NOT EXISTS PARTITION (ds = '2013_12_16') LOCATION '2013/12/16/';

      I get the following exception:

      java.io.IOException:org.apache.hadoop.hive.serde2.avro.BadSchemaException

      Attachments

        1. HIVE-6056.patch
          14 kB
          Ashish Singh

        Issue Links

        is related to

        Improvement - An improvement or enhancement to an existing feature or task. HIVE-8129 Umbrella JIRA to track Avro improvements

        • Major - Major loss of function.
        • Open
        relates to

        Bug - A problem which impairs or prevents the functions of the product. HIVE-3953 Reading of partitioned Avro data fails because of missing properties

        • Blocker - Blocks development and/or testing work, production could not run
        • Closed

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            singhashish Ashish Singh Assign to me
            rushilgupta Rushil Gupta
            Votes:
            2 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:

              Slack

                Issue deployment