Details
-
Improvement
-
Status: Resolved
-
Normal
-
Resolution: Fixed
-
None
-
Operability
-
Normal
-
All
-
None
-
Description
While CASSANDRA-11218 should also address this same problem, that ticket's a pretty big beautiful hammer and there's a fairly surgical modification we can make to the 2i building process to mitigate 2i building's ability to block compaction.
While 11218 approaches this from a "lots of compactions can starve 2i build and things never finish", what I'd like to address here is the inverse that building multiple secondary indices in parallel can occupy all available compactors to a point where we don't have any left to do actual compactions. Completing 2i build can take several hours and the backlog of compactions from that time may result in too many uncompacted sstables left to catch up.
A fairly simple fix is to break 2i building out to its own threadpool w/a hot proppable param you can use to change the # of threads for that build in the extreme cases where the default of 2 is indufficient.
Attachments
Issue Links
- relates to
-
CASSANDRA-19266 Make concurrent_index_builders configurable at runtime
- Resolved