Uploaded image for project: 'Mesos'
  1. Mesos
  2. MESOS-9852

Slow memory growth in master due to deferred deletion of offer filters and timers.

    XMLWordPrintableJSON

Details

    Description

      The allocator does not keep a handle to the offer filter timer, which means it cannot remove the timer overhead (in this case memory) when removing the offer filter earlier (e.g. due to revive):

      https://github.com/apache/mesos/blob/1.8.0/src/master/allocator/mesos/hierarchical.cpp#L1338-L1352

      In addition, the offer filter is allocated on the heap but not deleted until the timer fires (which might take forever!):

      https://github.com/apache/mesos/blob/1.8.0/src/master/allocator/mesos/hierarchical.cpp#L1321
      https://github.com/apache/mesos/blob/1.8.0/src/master/allocator/mesos/hierarchical.cpp#L1408-L1413
      https://github.com/apache/mesos/blob/1.8.0/src/master/allocator/mesos/hierarchical.cpp#L2249

      We'll need to try to backport this to all active release branches.

      Attachments

        1. statistics
          1.06 MB
          longfei
        2. screenshot-1.png
          74 kB
          longfei
        3. _tmp_libprocess.Do1MrG_profile 24hours.svg
          114 kB
          longfei
        4. _tmp_libprocess.Do1MrG_profile 24hours.dump
          58 kB
          longfei
        5. _tmp_libprocess.Do1MrG_profile (1).svg
          112 kB
          longfei
        6. _tmp_libprocess.Do1MrG_profile (1).dump
          46 kB
          longfei

        Activity

          People

            bmahler Benjamin Mahler
            bmahler Benjamin Mahler
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: