Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-15669

LeveledCompactionStrategy compact last level throw an ArrayIndexOutOfBoundsException

Agile BoardAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      Cassandra will throw an ArrayIndexOutOfBoundsException when compact last level.

      My test is as follows:

      1. Create a table with LeveledCompactionStrategy and its params are 'enabled': 'true', 'fanout_size': '2', 'max_threshold': '32', 'min_threshold': '4', 'sstable_size_in_mb': '2'(fanout_size and sstable_size_in_mb are too small just to make it easier to reproduce the problem);
      2. Insert data into the table by stress;
      3. Cassandra throw an ArrayIndexOutOfBoundsException when compact level9 sstables(this level score bigger than 1.001)

      ERROR [CompactionExecutor:4] 2020-03-28 08:59:00,990 CassandraDaemon.java:442 - Exception in thread Thread[CompactionExecutor:4,1,main]
      java.lang.ArrayIndexOutOfBoundsException: 9
      at org.apache.cassandra.db.compaction.LeveledManifest.getLevel(LeveledManifest.java:814)
      at org.apache.cassandra.db.compaction.LeveledManifest.getCandidatesFor(LeveledManifest.java:746)
      at org.apache.cassandra.db.compaction.LeveledManifest.getCompactionCandidates(LeveledManifest.java:398)
      at org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getNextBackgroundTask(LeveledCompactionStrategy.java:131)
      at org.apache.cassandra.db.compaction.CompactionStrategyHolder.lambda$getBackgroundTaskSuppliers$0(CompactionStrategyHolder.java:109)
      at org.apache.cassandra.db.compaction.AbstractStrategyHolder$TaskSupplier.getTask(AbstractStrategyHolder.java:66)
      at org.apache.cassandra.db.compaction.CompactionStrategyManager.getNextBackgroundTask(CompactionStrategyManager.java:214)
      at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:289)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
      at java.util.concurrent.FutureTask.run(FutureTask.java)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
      at java.lang.Thread.run(Thread.java:748)

      I tested it on cassandra version 3.11.3 & 4.0-alpha3. The exception all happened.

      once it triggers, level1- leveln compaction no longer works, level0 is still valid
       

      Attachments

        1. error_info.png
          37 kB
          sunhaihong
        2. cfs_compaction_info.png
          22 kB
          sunhaihong
        3. 15669-trunk.txt
          8 kB
          Aleksei Zotov
        4. 15669-4.0.txt
          8 kB
          Aleksei Zotov
        5. 15669-3.11.txt
          12 kB
          Aleksei Zotov

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            azotcsit Aleksei Zotov Assign to me
            sunhaihong sunhaihong
            Aleksei Zotov
            Marcus Eriksson, Yifan Cai
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 2h 10m
                2h 10m

                Slack

                  Issue deployment