Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-23553 Upgrade ORC version to 1.6.7
  3. HIVE-23036

ORC PPD eval with sub-millisecond timestamps

    XMLWordPrintableJSON

Details

    Description

      See ORC-611 for more details

      ORC stores timestamps with:

      • nanosecond precision for the data itself
      • milliseconds precision for min-max statistics

      As both min and max are rounded to the same value,  timestamps with ns precision will not pass the PPD evaluator.

      create table tsstat (ts timestamp) stored as orc;
      insert into tsstat values ("1970-01-01 00:00:00.0005");
      select * from tsstat where ts = "1970-01-01 00:00:00.0005";
      -- returned 0 rows

      ORC PPD evaluation currently happens as part of OrcInputFormat https://github.com/apache/hive/blob/7e39a2c13711f9377c9ce1edb4224880421b1ea5/ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java#L2314

      Attachments

        Issue Links

          Activity

            People

              pgaref Panagiotis Garefalakis
              pgaref Panagiotis Garefalakis
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 50m
                  1h 50m