Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-22731 MJ probe decode with row-level filtering
  3. HIVE-23882

Compiler extensions for MJ probe optimization

    XMLWordPrintableJSON

Details

    Description

      In probe we cannot currently support Key expressions (on the big table Side) as ORC CVs Probe directly the smalltable HT (there is no expr evaluation at that level).

      TezCompiler should take this into account when picking MJs to push probe details, In this ticket we extend probedecode compiler logic to:

      • Validate that MJ is a single Key join, where the bigTable keyCol is only a ExprNodeColumnDesc
      • For the selected MJ use backtracking logic to find the orignal TS key it maps to (on the same vertex) - this actually revealed several missed or wrong optimizations (updated q outs)
      • Finally, extend the optimization logic to check if there a type Cast between src and destination (as the types have to match in the probe case) and only use it on the LLAP mode for now

      Attachments

        Issue Links

          Activity

            People

              pgaref Panagiotis Garefalakis
              pgaref Panagiotis Garefalakis
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 3h 20m
                  3h 20m