Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-12899 Bring query optimization time down
  3. HIVE-13807

Extend metadata provider to pull up predicates through Union

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.1.0
    • 2.1.0
    • CBO
    • None

    Description

      Follow-up of HIVE-13068.

      Currently, when we pull up predicates through Union operation with Calcite metadata provider, we just create a single disjunction of pulled up predicates.

      E.g. Assume operators I1, I2, I3 with predicates P1, P2, P3 that can be pulled up through them, respectively.
      For an operation Union (I1, I2, I3) we infer a new predicate Pu, such that Pu = P1 OR P2 OR P3.
      While this is correct, we miss some chances for simplification e.g. if there are common factors in P1, P2, and P3. Further, this inference differs slightly from the way that the metadata provider pulls up predicates for other operators (a list of predicates), thus breaking some assumptions and missing some optimization opportunities.

      Ex. ql/src/test/results/clientpositive/input26.q.out

      explain
      select * from (
        select * from (select * from srcpart a where a.ds = '2008-04-08' and a.hr = '11' order by a.key limit 5)pa
          union all
        select * from (select * from srcpart b where b.ds = '2008-04-08' and b.hr = '14' limit 5)pb
      )subq
      

      Attachments

        1. HIVE-13807.01.patch
          40 kB
          jcamachorodriguez
        2. HIVE-13807.patch
          19 kB
          jcamachorodriguez

        Issue Links

          Activity

            People

              jcamacho Jesús Camacho Rodríguez
              jcamacho Jesús Camacho Rodríguez
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: