Details
-
Bug
-
Status: Resolved
-
Urgent
-
Resolution: Fixed
-
None
-
Critical
Description
I try to add a node in my cluster and doing some cleanup deleted all my data on a node. This makes the cluster totally broken since all next read seem to not be able to validate the data. Even a repair on the problematic node doesn't fix the issue. I've attached the bisect script used and the output results of the procedure.
Procedure to reproduce:
ccm stop && ccm remove ccm create -n 2 --install-dir=path/to/cassandra-2.1/branch demo ccm start ccm node1 stress -- write n=1000000 -schema replication\(factor=2\) -rate threads=50 ccm node1 nodetool status ccm add -i 127.0.0.3 -j 7400 node3 # no auto-boostrap ccm node3 start ccm node1 nodetool status ccm node3 repair ccm node3 nodetool status ccm node1 nodetool cleanup ccm node2 nodetool cleanup ccm node3 nodetool cleanup ccm node1 nodetool status ccm node1 repair ccm node1 stress -- read n=1000000 ## CRASH Data returned was not validated ?!?
bisec script output:
$ git bisect start cassandra-2.1 cassandra-2.1.3
$ git bisect run ~/dev/cstar/cleanup_issue/bisect.sh
...
4b05b204acfa60ecad5672c7e6068eb47b21397a is the first bad commit
commit 4b05b204acfa60ecad5672c7e6068eb47b21397a
Author: Benedict Elliott Smith <benedict@apache.org>
Date: Wed Feb 11 15:49:43 2015 +0000
Enforce SSTableReader.first/last
patch by benedict; reviewed by yukim for CASSANDRA-8744
:100644 100644 3f0463731e624cbe273dcb3951b2055fa5d9e1a2 b2f894eb22b9102d410f1eabeb3e11d26727fbd3 M CHANGES.txt
:040000 040000 51ac2a6cd39bd2377c2e1ed6693ef789ab65a26c 79fa2501f4155a64dca2bbdcc9e578008e4e425a M src
bisect run success