Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-2399

Cleanup/rethink QueryMaintenance() calls in the BE.

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • Impala 2.3.0
    • None
    • Backend

    Description

      In our BE execution, we currently call QueryMaintenance() and check for cancellation very generously, e.g., at the beginning of every GetNext() or in tight loops that could run for a long time (e.g. due to selective predicates).

      Our current pattern of usage has several problems:
      1. QueryMaintenance() is expensive, but the function name is rather opaque to this fact. It traverses the mem tracker hierarchy checking for the mem limit, and also checks the query status under a spin lock. It is not clear that these checks need to be coupled and called so frequently. We should place these checks judiciously/appropriately because QueryMaintenance() can become a performance bottleneck especially in subplan execution.

      2. We always couple QueryMaintenance() with checking for cancellation. There is an existing TODO in the code to address this (see runtime-state.cc RuntimeState::CheckQueryState()).

      See these code reviews for additional discussion:
      http://gerrit.cloudera.org:8080/894
      http://gerrit.cloudera.org:8080/901

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              alex.behm Alexander Behm
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated: