Details
-
Improvement
-
Status: Resolved
-
Normal
-
Resolution: Fixed
-
None
-
Operability
-
Normal
-
All
-
None
-
Description
In CASSANDRA-16663, we added a new rate limiter that applies to all incoming requests over the native transport. Given it has now had some time to marinate in 4.1 snapshot builds, there are a few things we should do to make sure it hits the bar for usability and observability:
1.) Add the native_transport_rate_limiting_enabled and native_transport_max_requests_per_second options to the example cassandra.yaml file w/ brief documentation.
2.) Add a simple Meter/metric, probably in ClientMetrics for the rate* of native transport requests dispatched for processing. Given there are a few cases where native transport requests may not translate 1:1 to the number of times we hit StorageProxy, having this metric should make verifying the correct operation of the limiter easier. (Ex. IN queries w/ multiple partition keys, GROUP BY queries, etc.) It may make sense to look further into adjusting the behavior of the rate limiter to take these kinds of queries into account, but that is a bit more complex than either of these 2 items, and deserves another Jira.
* We do expose a per-connection counter for the number of requests dispatched, and it is exposed over a virtual table and JMX, but even if it were global, it is not a rate metric.