Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-8721

Wrong result when Impala reads a Hive written parquet TimeStamp column

    XMLWordPrintableJSON

Details

    Description

       

      Easy to repro on latest upstream:

      hive> create table t1_hive(c1 timestamp) stored as parquet;
      hive> insert into t1_hive values('2009-03-09 01:20:03.600000000');
      hive> select * from t1_hive;
      OK
      2009-03-09 01:20:03.6
      [localhost:21000] default> invalidate metadata t1_hive;
      [localhost:21000] default> select * from t1_hive;
      Query: select * from t1_hive
      Query submitted at: 2019-06-24 09:55:36 (Coordinator: http://optimus-prime:25000)
      Query progress can be monitored at: http://optimus-prime:25000/query_plan?query_id=b34f85cb5da29c26:d4dfcb2400000000
      +-------------------------------+
      | c1 |
      +-------------------------------+
      | 2009-03-09 09:20:03.600000000 | <<<<<UTC
      +-------------------------------+
      bin/start-impala-cluster.py --impalad_args='-convert_legacy_hive_parquet_utc_timestamps=true'
      [localhost:21000] default> select * from t1_hive;
      Query: select * from t1_hive
      Query submitted at: 2019-06-24 10:00:22 (Coordinator: http://optimus-prime:25000)
      Query progress can be monitored at: http://optimus-prime:25000/query_plan?query_id=d5428bb21fb259b9:7b10703400000000
      +-------------------------------+
      | c1 |
      +-------------------------------+
      | 2009-03-09 02:20:03.600000000 |. <<<<<<PST8PDT
      +-------------------------------+
       
      

       

      This issue is causing testcase test_hive_impala_interop to fail. Untill this issue is fixed, the testcase will be updated to not include a timestamp column. The test case should be updated to include a timestamp column once this issue is fixed.

       

      Attachments

        Issue Links

          Activity

            People

              tarmstrong Tim Armstrong
              arawat Abhishek Rawat
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: