Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-14333

ORC schema evolution from float to double changes precision and breaks filters

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 2.1.0, 2.2.0
    • 2.1.1, 2.2.0
    • ORC
    • None

    Description

      ORC vs text schema evolution from float to double changes precision

      Text Schema Evolution
      hive> create table float_text(f float);
      hive> insert into float_text values(74.72);
      hive> select f from float_text;
      OK
      74.72
      hive> alter table float_text change column f f double;
      hive> select f from float_text;
      OK
      74.72
      
      Orc Schema Evolution
      hive> create table float_orc(f float) stored as orc;
      hive> insert into float_orc values(74.72);
      hive> select f from float_orc;
      OK
      74.72
      hive> alter table float_orc change column f f double;
      hive> select f from float_orc;
      OK
      74.72000122070312
      

      This will break all filters on the evolved column "f"

      Filter returning no results
      hive> set hive.optimize.index.filter=false;
      hive> select f from float_orc where f=74.72;
      OK
      

      Attachments

        1. HIVE-14333.1.patch
          14 kB
          Prasanth Jayachandran
        2. HIVE-14333.2.patch
          16 kB
          Prasanth Jayachandran
        3. HIVE-14333.3.patch
          31 kB
          Prasanth Jayachandran

        Issue Links

          Activity

            People

              prasanth_j Prasanth Jayachandran
              prasanth_j Prasanth Jayachandran
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: