Details
-
Bug
-
Status: Resolved
-
Normal
-
Resolution: Fixed
-
None
-
Availability - Unavailable
-
Normal
-
Normal
-
User Report
-
All
-
None
-
Description
If a node involved in a MOVE operation should fail, peers can sometimes maintain pending ranges for it even when it has left the ring and/or been replaced (in practice until the peer is next bounced). This in turn can lead to bogus unavailable responses to clients if a replica for the any of the pending ranges should go down.
If the moving node crashes hard, a subsequent replacement will correctly fail as long as cassandra.consistent.rangemovement is set to true because the new node will learn the MOVING status from the remaining peers. A graceful shutdown, however, causes that status to be replaced with SHUTDOWN, but doesn't update TokenMetadata, so pending ranges remain for the down node, even after it has been removed from the ring.