Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-3200 Replace BufferedBlockMgr with new buffer pool
  3. IMPALA-3204

Integrate buffer pool reservations with memtrackers

    XMLWordPrintableJSON

Details

    Description

      The reservation mechanism for the buffer pool introduces a memory tracking mechanism that is somewhat parallel to the existing MemTracker mechanism. This task is to integrate the two so that:

      • Query memory limits include buffer reservations
      • Runtime profiles and other diagnostics include buffer pool usage in the memory consumption, down to the exec node level.

      My initial thought is that each reservation tracker should optionally have an associated MemTracker. When reservations are granted, it would be counted towards the memory consumption of the tracker. I think we want to count reservations rather than usage, because the MemTrackers are really more geared towards tracking committed memory rather than used memory.

      Attachments

        Activity

          People

            tarmstrong Tim Armstrong
            tarmstrong Tim Armstrong
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: