Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-24963

Windowing expression may loose its input in some cases

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      drop table if exists sss;
       CREATE TABLE `sss`(
         `user_id` bigint,
         `user_mid` string
       )
       PARTITIONED BY (
         `dt` string)
      STORED AS ORC
         ;
      
      insert into sss partition(dt='part1') VALUES (12345,'user_mid v1'),(12345,'user_mid v1'),(12345,'user_mid v1'),(12345,'user_mid v1'),(12345,'user_mid v1');
      
      
      set hive.auto.convert.join.noconditionaltask.size=1;
      WITH
           unioned_user AS (
               SELECT
                   *,
                   row_number() OVER (PARTITION BY user_mid ORDER BY dt ASC) AS r_asc,
                   row_number() OVER (PARTITION BY user_mid ORDER BY dt DESC) AS r_desc
               FROM (
                   SELECT DISTINCT
                       dt,
                       user_mid
                   FROM sss
                   WHERE dt = '20210228'
                   UNION ALL
                   SELECT DISTINCT
                      dt,
                       user_mid
                   FROM sss
               ) AS uni
           ),
           merged_user AS (
               SELECT
                   a.user_mid
               FROM (SELECT * FROM unioned_user WHERE r_asc = 1) AS a
               INNER JOIN (SELECT * FROM unioned_user WHERE r_desc = 1) AS d
               ON a.user_mid = d.user_mid
           )
       Select count(*) from merged_user;
      

      Attachments

        1. mapjoin.png
          194 kB
          Zoltan Haindrich
        2. mergejoin.png
          151 kB
          Zoltan Haindrich

        Issue Links

          Activity

            People

              kgyrtkirk Zoltan Haindrich
              kgyrtkirk Zoltan Haindrich
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: