Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-12988

Calculate an unbounded version of CpuAsk

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • Impala 4.4.0
    • Frontend
    • None

    Description

      CpuAsk is calculated through recursive call beginning at Planner.computeBlockingAwareCores(), which called after Planner.computeEffectiveParallelism(). It does blocking operator analysis over selected degree of parallelism that decided during computeEffectiveParallelism() traversal. That selected degree of parallelism, however, is already bounded by min and max parallelism config, derived from PROCESSING_COST_MIN_THREADS and MAX_FRAGMENT_INSTANCES_PER_NODE options accordingly.

      It is beneficial to have another version of CpuAsk that is not bounded by min and max parallelism config. It should purely based on the fragment's ProcessingCost and query plan relationship constraint (ie., num JOIN BUILDER fragment should be equal as num JOIN fragment for partitioned join). During executor group set selection, Frontend should use the unbounded CpuAsk number to avoid assigning query to small executor group set prematurely.

      Attachments

        Activity

          People

            rizaon Riza Suminto
            rizaon Riza Suminto
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: