Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-21660

Wrong result when union all and later view with explode is used

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 3.1.1
    • 4.0.0-alpha-1
    • Physical Optimizer
    • None

    Description

      There is a data loss when the data is inserted to a partitioned table using union all and lateral view with explode. 

       

      Steps to reproduce:

       

      create table t1 (id int, dt string);
      insert into t1 values (2, '2019-04-01');
      create table t2( id int, dates array<string>);
      insert into t2 select 1 as id, array('2019-01-01','2019-01-02','2019-01-03') as dates;
      
      create table dst (id int) partitioned by (dt string);
      
      set hive.exec.dynamic.partition.mode=nonstrict;
      set hive.exec.dynamic.partition=true;
      insert overwrite table dst partition (dt)
      select t.id, t.dt from (
      select id, dt from t1
      union all
      select id, dts as dt from t2 tt2 lateral view explode(tt2.dates) dd as dts ) t;
      select * from dst;
      

       

       

      Actual Result:

      +--------------+--------------+
      | 2            | 2019-04-01   |
      +--------------+--------------+

       

      Expected Result (Run only the select part from the above insert query): 

      +-------+------------+
      | 2     | 2019-04-01 |
      | 1     | 2019-01-01 |
      | 1     | 2019-01-02 |
      | 1     | 2019-01-03 |
      +-------+------------+

       

      Data retrieved using union all and lateral view with explode from second table is missing. 

      Attachments

        1. HIVE-21660.patch
          1 kB
          Ganesha Shreedhara
        2. HIVE-21660.2.patch
          8 kB
          Ganesha Shreedhara
        3. HIVE-21660.1.patch
          9 kB
          Ganesha Shreedhara

        Issue Links

          Activity

            People

              ganeshas Ganesha Shreedhara
              ganeshas Ganesha Shreedhara
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: