Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-16754

Flaky o.a.c.distributed.test.SchemaTest

    XMLWordPrintableJSON

Details

    Description

      The JVM dtest org.apache.cassandra.distributed.test.SchemaTest is flaky:
      https://ci-cassandra.apache.org/job/Cassandra-4.0.0/34/testReport/junit/org.apache.cassandra.distributed.test/SchemaTest/readRepairWithCompaction_2/

      Error Message
      FSWriteError in /home/cassandra/cassandra/tmp/dtests5018452609443646984/node2/commitlog/CommitLog-7-1624286958980.log
      Stacktrace
      java.lang.RuntimeException: FSWriteError in /home/cassandra/cassandra/tmp/dtests5018452609443646984/node2/commitlog/CommitLog-7-1624286958980.log
      	at org.apache.cassandra.distributed.impl.Instance.lambda$startup$10(Instance.java:582)
      	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
      	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
      	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
      	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
      	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
      	at java.base/java.lang.Thread.run(Thread.java:834)
      Caused by: FSWriteError in /home/cassandra/cassandra/tmp/dtests5018452609443646984/node2/commitlog/CommitLog-7-1624286958980.log
      	at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:256)
      	at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:273)
      	at org.apache.cassandra.db.commitlog.AbstractCommitLogSegmentManager.handleReplayedSegment(AbstractCommitLogSegmentManager.java:349)
      	at org.apache.cassandra.db.commitlog.CommitLog.recoverSegmentsOnDisk(CommitLog.java:178)
      	at org.apache.cassandra.distributed.impl.Instance.lambda$startup$10(Instance.java:508)
      Caused by: java.nio.file.NoSuchFileException: /home/cassandra/cassandra/tmp/dtests5018452609443646984/node2/commitlog/CommitLog-7-1624286958980.log
      	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
      	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
      	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
      	at java.base/sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:249)
      	at java.base/sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:105)
      	at java.base/java.nio.file.Files.delete(Files.java:1142)
      	at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:250)
      

      Although it doesn't fail frequently on CI it's quite easy to reproduce it locally.

      I think the failure is caused by the two tests on the class not waiting for the future returned by IInstance#shutdown().

      Attachments

        Issue Links

          Activity

            People

              adelapena Andres de la Peña
              adelapena Andres de la Peña
              Andres de la Peña
              Andres de la Peña, Berenguer Blasi
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m