Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-2506

Account for memory usage when preparing fragment

    XMLWordPrintableJSON

Details

    Description

      Preparing a query fragment for execution can consume a significant amount of untracked memory, between plan fragments, descriptors, and in-memory LLVM structures. This is problematic if we're at or near the process memory limit, since it will go further over the limit. This memory isn't freed until the fragment is fully canceled and deregistered.

      In this situation better behaviour would be to not start the fragment and notify the coordinator that the fragment was cancelled immediately to avoid consuming resources unnecessarily.

      Attachments

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. IMPALA-3290 Catch std::bad_alloc from LLVM optimisation

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. IMPALA-3091 LLVM memory is not accounted for in query mem tracker

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. IMPALA-2120 Track mem of std structures in query execution

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Activity

            People

              tarmstrong Tim Armstrong
              tarmstrong Tim Armstrong
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: