Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-6271

Replace SnapTree in AtomicSortedColumns

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Normal
    • Resolution: Fixed
    • 2.1 beta1
    • None

    Description

      On the write path a huge percentage of time is spent in GC (>50% in my tests, if accounting for slow down due to parallel marking). SnapTrees are both GC unfriendly due to their structure and also very expensive to keep around - each column name in AtomicSortedColumns uses > 100 bytes on average (excluding the actual ByteBuffer).

      I suggest using a sorted array; changes are supplied at-once, as opposed to one at a time, and if < 10% of the keys in the array change (and data equal to < 10% of the size of the key array) we simply overlay a new array of changes only over the top. Otherwise we rewrite the array. This method should ensure much less GC overhead, and also save approximately 80% of the current memory overhead.

      TreeMap is similarly difficult object for the GC, and a related task might be to remove it where not strictly necessary, even though we don't keep them hanging around for long. TreeMapBackedSortedColumns, for instance, seems to be used in a lot of places where we could simply sort the columns.

      Attachments

        1. 0001-Always-call-ReplaceFunction.txt
          4 kB
          Sylvain Lebresne
        2. oprate.svg
          0.6 kB
          Benedict Elliott Smith
        3. tmp.patch
          0.9 kB
          Benedict Elliott Smith
        4. tmp2.patch
          7 kB
          Benedict Elliott Smith
        5. tmp3.patch
          7 kB
          Benedict Elliott Smith

        Activity

          People

            benedict Benedict Elliott Smith
            benedict Benedict Elliott Smith
            Benedict Elliott Smith
            Jonathan Ellis
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: