Infinite compaction of L0 SSTables in JBOD

I recently upgraded from 2.2.6 to 3.11.0.

I am seeing Cassandra loop infinitely compacting the same data over and over. Attaching logs.

It is compacting two tables, one on /srv/disk10, the other on /srv/disk1. It does create new SSTables but immediately recompacts again. Note that I am not inserting anything at the moment, there is no flushing happening on this table (Memtable switch count has not changed).

My theory is that it somehow thinks those should be compaction candidates. But they shouldn't be, they are on different disks and I ran nodetool relocatesstables as well as nodetool compact. So, it tries to compact them together, but the compaction results in the exact same 2 SSTables on the 2 disks, because the keys are split by data disk.

This is pretty serious, because all our nodes right now are consuming CPU doing this for multiple tables, it seems.