Uploaded image for project: 'Jackrabbit Oak'
  1. Jackrabbit Oak
  2. OAK-4865

Test failure: HeavyWriteIT

    XMLWordPrintableJSON

Details

    Description

      Said test sometimes fails with the following stack trace:

      09:46:40.900 ERROR [main] SegmentId.java:127                Segment not found: 8399230c-9338-47e3-acf5-b92d326cf171. SegmentId age=7473ms,gc-count=32,gc-status=success,store-generation=29,reclaim-predicate=(generation<=27),segment-generation27
      org.apache.jackrabbit.oak.segment.SegmentNotFoundException: Segment 8399230c-9338-47e3-acf5-b92d326cf171 not found
      at org.apache.jackrabbit.oak.segment.file.FileStore$14.call(FileStore.java:1345) ~[oak-segment-tar-0.0.13-SNAPSHOT.jar:na]
      at org.apache.jackrabbit.oak.segment.file.FileStore$14.call(FileStore.java:1285) ~[oak-segment-tar-0.0.13-SNAPSHOT.jar:na]
      at org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.load(CacheLIRS.java:1013) ~[oak-core-1.5.8.jar:1.5.8]
      at org.apache.jackrabbit.oak.cache.CacheLIRS$Segment.get(CacheLIRS.java:974) ~[oak-core-1.5.8.jar:1.5.8]
      at org.apache.jackrabbit.oak.cache.CacheLIRS.get(CacheLIRS.java:285) ~[oak-core-1.5.8.jar:1.5.8]
      at org.apache.jackrabbit.oak.segment.SegmentCache.getSegment(SegmentCache.java:92) ~[oak-segment-tar-0.0.13-SNAPSHOT.jar:na]
      at org.apache.jackrabbit.oak.segment.file.FileStore.readSegment(FileStore.java:1285) ~[oak-segment-tar-0.0.13-SNAPSHOT.jar:na]
      at org.apache.jackrabbit.oak.segment.SegmentId.getSegment(SegmentId.java:123) ~[oak-segment-tar-0.0.13-SNAPSHOT.jar:na]
      at org.apache.jackrabbit.oak.segment.Record.getSegment(Record.java:70) [oak-segment-tar-0.0.13-SNAPSHOT.jar:na]
      at org.apache.jackrabbit.oak.segment.SegmentNodeState.getStableIdBytes(SegmentNodeState.java:139) [oak-segment-tar-0.0.13-SNAPSHOT.jar:na]
      at org.apache.jackrabbit.oak.segment.SegmentNodeState.getStableId(SegmentNodeState.java:122) [oak-segment-tar-0.0.13-SNAPSHOT.jar:na]
      at org.apache.jackrabbit.oak.segment.SegmentNodeState.fastEquals(SegmentNodeState.java:633) [oak-segment-tar-0.0.13-SNAPSHOT.jar:na]
      at org.apache.jackrabbit.oak.segment.SegmentNodeStore$Commit.execute(SegmentNodeStore.java:604) [oak-segment-tar-0.0.13-SNAPSHOT.jar:na]
      at org.apache.jackrabbit.oak.segment.SegmentNodeStore.merge(SegmentNodeStore.java:265) [oak-segment-tar-0.0.13-SNAPSHOT.jar:na]
      at org.apache.jackrabbit.oak.segment.HeavyWriteIT.heavyWrite(HeavyWriteIT.java:85) [test-classes/:na]
      

      This is a problem with the test, not a regression:

      Segment not found: 8399230c-9338-47e3-acf5-b92d326cf171. SegmentId age=7473ms,gc-count=32,gc-status=success,store-generation=29,reclaim-predicate=(generation<=27),segment-generation27
      

      This means the missing segment was successfully gc'ed at GC #32. Its generation was 27 while the store just got bumped to generation 29. This causes a cleanup of all generations <= 27.

      The test itself calls FileStore.gc() in quick succession while at the same time writing to the store. This is likely to at some point cause a write to be based on an already collected segment. I suggest to fix this by increasing the number of retained generations to a sufficiently high value (for this test).

      On a side node, this issue (and being able to to a root cause analysis) validates the additional logging that we added with OAK-2405!

      Attachments

        Activity

          People

            mduerig Michael Dürig
            mduerig Michael Dürig
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: