Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-2798

After adding a column to avro table, Impala returns weird result if codegen is enabled.

    XMLWordPrintableJSON

Details

    Description

      Step to reproduce:
      in Hive
      1. create an partitioned avro table, "avro1"
      2. insert some data with snappy compression
      3. add a column to the table
      4. insert more data including new column
      5. create a second table "avro2" with the old table schema (not include the new added column)
      6. msck repair table "avro2"
      In Impala
      7. invalidate metadata and query table "avro2", got weird data at the end.
      8. disable codegen and query table again, it shows proper result.

      seems like when codegen enabled, it gets row offset wrong from the data with extra column.

      sometimes impalad hits DCHECK and crashes.

      F1222 16:12:23.601328 21930 read-write-util.cc:29] Check failed: shift <= 64 (70 vs. 64) 
      *** Check failure stack trace: ***
          @          0x21d471d  google::LogMessage::Fail()
          @          0x21d6ebf  google::LogMessage::SendToLog()
          @          0x21d4287  google::LogMessage::Flush()
          @          0x21d774d  google::LogMessageFatal::~LogMessageFatal()
          @          0x16ce213  impala::ReadWriteUtil::ReadZLong()
          @     0x7f00f3f96593  (unknown)
      

      Attachments

        Activity

          People

            jyu@cloudera.com Juan Yu
            jyu@cloudera.com Juan Yu
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: