Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-18454

Enhance nodetool gcstats

Agile BoardAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsAdd voteVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Normal
    • Resolution: Unresolved
    • None
    • Tool/nodetool
    • None
    • Operability
    • Low Hanging Fruit
    • All
    • None

    Description

      Nodetool gcstats provides only a brief one line output of historical GC activity with Max, Total, Stdev, etc

      % nodetool gcstats
             Interval (ms) Max GC Elapsed (ms)Total GC Elapsed (ms)Stdev GC Elapsed (ms)   GC Reclaimed (MB)         Collections      Direct Memory Bytes
                     11979                   0                   0                 NaN                   0                   0                       -1
      

      1. Performance of garbage collection is such an key part of Cassandra performance that it would be helpful to have more complete metrics with gcstats here.  The MBean java.lang.GarbageCollector has recent statistic which could be added in events/second for:

      • GcThreadCount
      • Duration

      The current size of eden / survivor / old space and humongous allocations should be shown as well.

      The metrics show with jhsdb jmap would be very helpful as well, if they're available.

      https://docs.oracle.com/en/java/javase/11/troubleshoot/diagnostic-tools.html#GUID-A69901EC-F87D-4B63-A8B7-DE8684AD4FF9

      2. Since physical memory is also a critical part of GC performance, it would be useful to report the following MBeans 

      cassandra_os_free_memory_bytes Amount of free physical memory available (as seen by the Cassandra JVM process).
      cassandra_os_free_swap_bytes Amount of free swap space available (as seen by the Cassandra JVM process).
      cassandra_os_memory_bytes_total Total physical memory available (as seen by the Cassandra JVM process).
      cassandra_os_swap_bytes_total Total swap space available (as seen by the Cassandra JVM process).

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned Assign to me
            bschoeni Brad Schoening

            Dates

              Created:
              Updated:

              Slack

                Issue deployment