Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-16261

Prevent unbounded number of flushing tasks

    XMLWordPrintableJSON

Details

    Description

      The cleaner thread is not prevented from queueing an unbounded number of flushing tasks for memtables that are almost empty.

      This patch adds a mechanism to track the number of pending flushing
      tasks in the memtable cleaner. Above the maximum number (2x the flushing
      threads by default), only memtables using at least MCT memory will be
      flushed, where MCT stands for Memory Cleanup Threshold.

      This patch also fixes a possible problem tracking the memory marked as
      "reclaiming" in the memtable allocators and pool. Writes that complete
      only after a memtable has been scheduled for flushing, did not report
      their memory as reclaiming. Normally this should be a small value of no
      consequence, but if the flushing tasks are blocked for a long period,
      and there is a sufficient number of writes, or these writes use
      a sufficiently large quantity of memory, this would cause the memtable
      cleaning algorithm to schedule repeated flushing tasks because the used
      memory is always > reclaiming memory + MCT.

      Attachments

        Issue Links

          Activity

            People

              e.dimitrova Ekaterina Dimitrova
              e.dimitrova Ekaterina Dimitrova
              Ekaterina Dimitrova
              Andres de la Peña, Caleb Rackliffe
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: