Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-21419

Partition Pruning not happening when using Apache Ranger masking

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.3.2
    • 3.0.0
    • None

    Description

      I have a partitioned table, which I have a Ranger masking policy on a non-partition column. When I am attempting to query the table that includes the column that has masking enabled, then partition pruning no longer occurs. 

      To reproduce:

      Create two partitioned tables. I used TPC-H tables as they are publicly available and will provide the schemas and queries I used. Insert into the second table from the first table. For example:

      insert overwrite table analyst1.test1 PARTITION (l_shipdate)
      select l_commitdate, l_receiptdate, l_shipdate
      from analyst1.lineitem_partitioned
      where l_shipdate = '1992-01-02';

      I have attached the explain plan when a masking rule on l_commitdate is enabled and when not enabled.

      I have done a bit of deep dive and see that the pruning expression is not being set when the masking rule is enabled. 

      Attachments

        1. hive-jira-schema-explain-plan.txt
          42 kB
          Mert Hocanin
        2. Operators-in-debugger-with-masking.png
          57 kB
          Mert Hocanin
        3. Operators-in-debugger-without-masking.png
          59 kB
          Mert Hocanin

        Issue Links

          Activity

            People

              Unassigned Unassigned
              mert_hoc Mert Hocanin
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: