Details
Description
Since Cassandra 2.1, streaming sessions are started by running a StreamSessionConnector task for each session in a dedicated executor (a static field of StreamCoordinator).
That executor is initialized with DebuggableThreadPoolExecutor.createWithFixedPoolSize, which means that once created (up to the given limit of the number of logical cores), its threads are kept alive for Integer.MAX_VALUE seconds.
This practically means that once a node needs to establish streaming sessions to n other nodes, it will create Math.min(n, numLogicalCores) StreamConnectionEstablisher threads that will stay parked forever after initializing (not completing) the session.
It seems preferable to replace DebuggableThreadPoolExecutor.createWithFixedPoolSize with DebuggableThreadPoolExecutor.createWithMaximumPoolSize which allows providing a saner keep-alive period (e.g. a minute).
That's also what createWithFixedPoolSize's Javadoc recommends: If (most) threads are expected to be idle most of the time, prefer createWithMaxSize() instead.