Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5073

Considering bypassing TCMalloc by default for buffer pool

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • Impala 2.9.0
    • None
    • Backend

    Description

      There would be some advantages to switch from from allocating buffers via TCMalloc and instead using mmap directly - e.g. less contention for the page heap lock.

      There are also downsides - virtual memory consumption could increase and we may end up mapping and unmapping memory more frequently.

      We would also need to wire up the MemTrackers so they include this memory in the process estimate.

      Attachments

        Issue Links

          is blocked by

          Sub-task - The sub-task of the issue IMPALA-3203 Implement scalable buffer recycling in buffer pool

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is depended upon by

          Bug - A problem which impairs or prevents the functions of the product. IMPALA-2800 impalad process blocked while releasing memory after a big query

          • Major - Major loss of function.
          • Resolved
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. IMPALA-5229 Try using TCMalloc + Huge Pages for buffers

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              Unassigned Unassigned
              tarmstrong Tim Armstrong
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated: