Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-6517 Snapshot support for Ozone
  3. HDDS-8557

Cannot reuse MiniOzoneCluster dir due to RocksDBCheckpointDiffer instance cache

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.4.0
    • 1.4.0
    • test

    Description

      RocksDBCheckpointDiffer has single instance per DB, which may cause failure if MiniOzoneCluster reuses the same directory between runs. Apparently this can happen if cluster is started in @BeforeAll method and test is repeated due to @Flaky annotation. The result is that if the first run fails, repeated attempts always run into the error.

      org.apache.hadoop.ozone.client.rpc.TestOzoneRpcClient  Time elapsed: 0.649 s  <<< ERROR!
      java.lang.RuntimeException: Failed to append compaction log to /home/runner/work/ozone/ozone/hadoop-ozone/integration-test/target/test-dir/MiniOzoneClusterImpl-7cd5347c-ea1a-48ee-af78-2d3d86f029cd/ozone-meta/db.snapshots/diffState/compaction-log/0000000000000000000.log
      	at org.apache.ozone.rocksdiff.RocksDBCheckpointDiffer.appendToCurrentCompactionLog(RocksDBCheckpointDiffer.java:381)
      	at org.apache.ozone.rocksdiff.RocksDBCheckpointDiffer.setCurrentCompactionLog(RocksDBCheckpointDiffer.java:314)
      	at org.apache.hadoop.hdds.utils.db.RDBStore.<init>(RDBStore.java:155)
      	at org.apache.hadoop.hdds.utils.db.DBStoreBuilder.build(DBStoreBuilder.java:212)
      	at org.apache.hadoop.ozone.om.OmMetadataManagerImpl.loadDB(OmMetadataManagerImpl.java:558)
      	at org.apache.hadoop.ozone.om.OmMetadataManagerImpl.loadDB(OmMetadataManagerImpl.java:526)
      	at org.apache.hadoop.ozone.om.OmMetadataManagerImpl.start(OmMetadataManagerImpl.java:516)
      	at org.apache.hadoop.ozone.om.OmMetadataManagerImpl.<init>(OmMetadataManagerImpl.java:325)
      	at org.apache.hadoop.ozone.om.OzoneManager.instantiateServices(OzoneManager.java:768)
      	at org.apache.hadoop.ozone.om.OzoneManager.<init>(OzoneManager.java:648)
      	at org.apache.hadoop.ozone.om.OzoneManager.createOm(OzoneManager.java:733)
      	at org.apache.hadoop.ozone.MiniOzoneClusterImpl$Builder.createOM(MiniOzoneClusterImpl.java:811)
      	at org.apache.hadoop.ozone.MiniOzoneClusterImpl$Builder.build(MiniOzoneClusterImpl.java:603)
      	at org.apache.hadoop.ozone.client.rpc.TestOzoneRpcClientAbstract.startCluster(TestOzoneRpcClientAbstract.java:211)
      	at org.apache.hadoop.ozone.client.rpc.TestOzoneRpcClient.init(TestOzoneRpcClient.java:51)
      ...
      Caused by: java.nio.file.NoSuchFileException: /home/runner/work/ozone/ozone/hadoop-ozone/integration-test/target/test-dir/MiniOzoneClusterImpl-7cd5347c-ea1a-48ee-af78-2d3d86f029cd/ozone-meta/db.snapshots/diffState/compaction-log/0000000000000000000.log
      

      Attachments

        Issue Links

          Activity

            People

              dteng Dave Teng
              adoroszlai Attila Doroszlai
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: