When we use the TRUNCATE command to delete all data in the table, the deleted data come back after a node restart or upgrade. This problem happens at the latest releases (2.2.19, 3.0.28, or 4.0.7)

Steps to reproduce

To reproduce it at release (3.0.28 or 4.0.7)

Start up a single Cassandra node. Using the default configuration and execute the following cqlsh commands.

CREATE KEYSPACE IF NOT EXISTS ks WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };
CREATE TABLE  ks.tb (c3 TEXT,c4 TEXT,c2 INT,c1 TEXT, PRIMARY KEY (c1, c2, c3 ));
INSERT INTO ks.tb (c3, c1, c2) VALUES ('val1','val2',1);
CREATE INDEX IF NOT EXISTS tb ON ks.tb ( c3);
TRUNCATE TABLE ks.tb;
DROP INDEX IF EXISTS ks.tb; 

Execute a read command

cqlsh> SELECT c2 FROM ks.tb; 

 c2
----

(0 rows) 

Then, we flush the node and kill the Cassandra daemon by

bin/nodetool flush
pgrep -f cassandra | xargs kill -9 

We restart the node. When the node has started, perform the same read, and the deleted data comes back again.

cqlsh> SELECT c2 FROM ks.tb; 

 c2
----
  1

(1 rows) 

To reproduce it at release (2.2.19)

We don't need to kill the Cassandra daemon. Use bin/nodetool stopdaemon is enough. The other steps are the same as reproducing it at 4.0.7 or 3.0.28.

bin/nodetool -h ::FFFF:127.0.0.1 flush 
bin/nodetool -h ::FFFF:127.0.0.1 stopdaemon

I have put the full log to reproduce it for release 4.0.7 and 2.2.19 in the comments.