Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-34495

Resuming Savepoint (rocks, scale up, heap timers) end-to-end test failure due to FileNotFoundException

    XMLWordPrintableJSON

Details

    Description

      https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=57760&view=logs&j=e9d3d34f-3d15-59f4-0e3e-35067d100dfe&t=5d91035e-8022-55f2-2d4f-ab121508bf7e&l=2010

      java.util.concurrent.ExecutionException: java.io.IOException: Could not open output stream for state backend
              at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:?]
              at java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:?]
              at org.apache.flink.util.concurrent.FutureUtils.runIfNotDoneAndGet(FutureUtils.java:511) ~[flink-dist-1.20-SNAPSHOT.jar:1.20-SNAPSHOT]
              at org.apache.flink.streaming.api.operators.OperatorSnapshotFinalizer.<init>(OperatorSnapshotFinalizer.java:54) ~[flink-dist-1.20-SNAPSHOT.jar:1.20-SNAPSHOT]
              at org.apache.flink.streaming.runtime.tasks.AsyncCheckpointRunnable.finalizeNonFinishedSnapshots(AsyncCheckpointRunnable.java:191) ~[flink-dist-1.20-SNAPSHOT.jar:1.20-SNAPSHOT]
              at org.apache.flink.streaming.runtime.tasks.AsyncCheckpointRunnable.run(AsyncCheckpointRunnable.java:124) [flink-dist-1.20-SNAPSHOT.jar:1.20-SNAPSHOT]
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
              at java.lang.Thread.run(Thread.java:829) [?:?]
      Caused by: java.io.IOException: Could not open output stream for state backend
              at org.apache.flink.runtime.state.filesystem.FsCheckpointStreamFactory$FsCheckpointStateOutputStream.createStream(FsCheckpointStreamFactory.java:461) ~[flink-dist-1.20-SNAPSHOT.jar:1.20-SNAPSHOT]
              at org.apache.flink.runtime.state.filesystem.FsCheckpointStreamFactory$FsCheckpointStateOutputStream.flushToFile(FsCheckpointStreamFactory.java:308) ~[flink-dist-1.20-SNAPSHOT.jar:1.20-SNAPSHOT]
              at org.apache.flink.runtime.state.filesystem.FsCheckpointStreamFactory$FsCheckpointStateOutputStream.write(FsCheckpointStreamFactory.java:284) ~[flink-dist-1.20-SNAPSHOT.jar:1.20-SNAPSHOT]
              at org.apache.flink.contrib.streaming.state.RocksDBStateUploader.uploadLocalFileToCheckpointFs(RocksDBStateUploader.java:148) ~[flink-dist-1.20-SNAPSHOT.jar:1.20-SNAPSHOT]
              at org.apache.flink.contrib.streaming.state.RocksDBStateUploader.lambda$createUploadFutures$0(RocksDBStateUploader.java:111) ~[flink-dist-1.20-SNAPSHOT.jar:1.20-SNAPSHOT]
              at org.apache.flink.util.function.CheckedSupplier.lambda$unchecked$0(CheckedSupplier.java:32) ~[flink-dist-1.20-SNAPSHOT.jar:1.20-SNAPSHOT]
              at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700) ~[?:?]
              ... 3 more
      Caused by: java.io.FileNotFoundException: /home/vsts/work/1/s/flink-end-to-end-tests/test-scripts/temp-test-directory-32462964103/savepoint-e2e-test-chckpt-dir/3c9ffc670ead2cb3c4118410cbef3b72/chk-12/3415a2f2-b0c8-4a07-b4a1-bb6cc58a7c56 (No such file or directory)
              at java.io.FileOutputStream.open0(Native Method) ~[?:?]
              at java.io.FileOutputStream.open(FileOutputStream.java:298) ~[?:?]
              at java.io.FileOutputStream.<init>(FileOutputStream.java:237) ~[?:?]
              at java.io.FileOutputStream.<init>(FileOutputStream.java:187) ~[?:?]
              at org.apache.flink.core.fs.local.LocalDataOutputStream.<init>(LocalDataOutputStream.java:50) ~[flink-dist-1.20-SNAPSHOT.jar:1.20-SNAPSHOT]
              at org.apache.flink.core.fs.local.LocalFileSystem.create(LocalFileSystem.java:266) ~[flink-dist-1.20-SNAPSHOT.jar:1.20-SNAPSHOT]
              at org.apache.flink.core.fs.SafetyNetWrapperFileSystem.create(SafetyNetWrapperFileSystem.java:130) ~[flink-dist-1.20-SNAPSHOT.jar:1.20-SNAPSHOT]
              at org.apache.flink.core.fs.EntropyInjector.createEntropyAware(EntropyInjector.java:76) ~[flink-dist-1.20-SNAPSHOT.jar:1.20-SNAPSHOT]
              at org.apache.flink.runtime.state.filesystem.FsCheckpointStreamFactory$FsCheckpointStateOutputStream.createStream(FsCheckpointStreamFactory.java:451) ~[flink-dist-1.20-SNAPSHOT.jar:1.20-SNAPSHOT]
              at org.apache.flink.runtime.state.filesystem.FsCheckpointStreamFactory$FsCheckpointStateOutputStream.flushToFile(FsCheckpointStreamFactory.java:308) ~[flink-dist-1.20-SNAPSHOT.jar:1.20-SNAPSHOT]
              at org.apache.flink.runtime.state.filesystem.FsCheckpointStreamFactory$FsCheckpointStateOutputStream.write(FsCheckpointStreamFactory.java:284) ~[flink-dist-1.20-SNAPSHOT.jar:1.20-SNAPSHOT]
              at org.apache.flink.contrib.streaming.state.RocksDBStateUploader.uploadLocalFileToCheckpointFs(RocksDBStateUploader.java:148) ~[flink-dist-1.20-SNAPSHOT.jar:1.20-SNAPSHOT]
              at org.apache.flink.contrib.streaming.state.RocksDBStateUploader.lambda$createUploadFutures$0(RocksDBStateUploader.java:111) ~[flink-dist-1.20-SNAPSHOT.jar:1.20-SNAPSHOT]
              at org.apache.flink.util.function.CheckedSupplier.lambda$unchecked$0(CheckedSupplier.java:32) ~[flink-dist-1.20-SNAPSHOT.jar:1.20-SNAPSHOT]
              at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1700) ~[?:?]
      

      Attachments

        Issue Links

          Activity

            People

              zakelly Zakelly Lan
              mapohl Matthias Pohl
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: