Uploaded image for project: 'Jackrabbit Oak'
  1. Jackrabbit Oak
  2. OAK-2761

Persistent cache: add data in a different thread

    XMLWordPrintableJSON

Details

    Description

      The persistent cache usually stores data in a background thread, but sometimes (if a lot of data is added quickly) the foreground thread is blocked.

      Even worse, switching the cache file can happen in a foreground thread, with the following stack trace.

      "127.0.0.1 [1428931262206] POST /bin/replicate.json HTTP/1.1" prio=5 tid=0x00007fe5df819800 nid=0x9907 runnable [0x0000000113fc4000]
   java.lang.Thread.State: RUNNABLE
        ...
	at org.h2.mvstore.MVStoreTool.compact(MVStoreTool.java:404)
	at org.apache.jackrabbit.oak.plugins.document.persistentCache.PersistentCache$1.closeStore(PersistentCache.java:213)
	- locked <0x0000000782483050> (a org.apache.jackrabbit.oak.plugins.document.persistentCache.PersistentCache$1)
	at org.apache.jackrabbit.oak.plugins.document.persistentCache.PersistentCache.switchGenerationIfNeeded(PersistentCache.java:350)
	- locked <0x0000000782455710> (a org.apache.jackrabbit.oak.plugins.document.persistentCache.PersistentCache)
	at org.apache.jackrabbit.oak.plugins.document.persistentCache.NodeCache.write(NodeCache.java:85)
	at org.apache.jackrabbit.oak.plugins.document.persistentCache.NodeCache.put(NodeCache.java:130)
	at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore.applyChanges(DocumentNodeStore.java:1060)
	at org.apache.jackrabbit.oak.plugins.document.Commit.applyToCache(Commit.java:599)
	at org.apache.jackrabbit.oak.plugins.document.CommitQueue.afterTrunkCommit(CommitQueue.java:127)
	- locked <0x0000000781890788> (a org.apache.jackrabbit.oak.plugins.document.CommitQueue)
	at org.apache.jackrabbit.oak.plugins.document.CommitQueue.done(CommitQueue.java:83)
	at org.apache.jackrabbit.oak.plugins.document.DocumentNodeStore.done(DocumentNodeStore.java:637)

      To avoid blocking the foreground thread, one solution is to store all data in a separate thread. If there is too much data added, then some of the data is not stored. If possible, the data that was not referenced a lot, and / or old revisions of documents (if new revisions are available).

      Attachments

        1. OAK-2761-trunk.patch
          49 kB
          Tomek Rękawek
        2. OAK-2761-1.2.patch
          45 kB
          Tomek Rękawek
        3. AsyncCacheTest.patch
          3 kB
          Marcel Reutegger

        Issue Links

          blocks

          Epic - Created by Jira Software - do not edit or delete. Issue type for a big user story that needs to be broken down. OAK-3286 Persistent Cache improvements

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is duplicated by

          Improvement - An improvement or enhancement to an existing feature or task. OAK-2686 Persistent cache: log activity and timing data, and possible optimizations

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. OAK-2745 PersistentCache should rely on eviction callback to add entry to the persistent cache

          • Major - Major loss of function.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. OAK-4882 Bottleneck in the asynchronous persistent cache

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. OAK-7559 CacheActionDispatcher not memory bound

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. OAK-2815 Persistent cache may block commit

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. OAK-4123 Persistent cache: allow to configure the add data concurrency

          • Major - Major loss of function.
          • Closed
          requires

          Improvement - An improvement or enhancement to an existing feature or task. OAK-3997 Include eviction cause to the LIRS removal callback

          • Major - Major loss of function.
          • Closed

          Activity

            People

              thomasm Thomas Mueller
              thomasm Thomas Mueller
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: