Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-13215

Cassandra nodes startup time 20x more after upgarding to 3.x

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Normal
    • Resolution: Fixed
    • 3.11.2, 4.0-alpha1, 4.0
    • Legacy/Core
    • None
    • Cluster setup: two datacenters (dc-main, dc-backup).
      dc-main - 9 servers, no vnodes
      dc-backup - 6 servers, vnodes

    Description

      CompactionStrategyManage.getCompactionStrategyIndex is called on each sstable at startup. And this function calls StorageService.getDiskBoundaries. And getDiskBoundaries calls AbstractReplicationStrategy.getAddressRanges.
      It appears that last function can be really slow. In our environment we have 1545 tokens and with NetworkTopologyStrategy it can make 1545*1545 computations in worst case (maybe I'm wrong, but it really takes lot's of cpu).

      Also this function can affect runtime later, cause it is called not only during startup.

      I've tried to implement simple cache for getDiskBoundaries results and now startup time is about one minute instead of 25m, but I'm not sure if it's a good solution.

      Attachments

        1. simple-cache.patch
          3 kB
          Viktor Kuzmin

        Issue Links

          Activity

            People

              marcuse Marcus Eriksson
              kvaster Viktor Kuzmin
              Marcus Eriksson
              Paulo Motta
              Votes:
              3 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: