Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-28458

BucketCache.notifyFileCachingCompleted may incorrectly consider a file fully cached

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      Noticed that TestBucketCachePersister.testPrefetchBlockEvictionWhilePrefetchRunning was flakey, failing whenever the block eviction happened while prefetch was still ongoing.

      In the test, we pass an instance of BucketCache directly to the cache config, so the test is actually placing both data and meta blocks in the bucket cache. So sometimes, the test call BucketCache.notifyFileCachingCompleted after the it has already evicted two blocks.  

      Inside BucketCache.notifyFileCachingCompleted, we iterate through the backingMap entry set, counting number of blocks for the given file. Then, to consider whether the file is fully cached or not, we do the following validation:

      if (dataBlockCount == count.getValue() || totalBlockCount == count.getValue()) {
        LOG.debug("File {} has now been fully cached.", fileName);
        fileCacheCompleted(fileName, size);
      }  

      But the test generates 57 total blocks, 55 data and 2 meta blocks. It evicts two blocks and asserts that the file hasn't been considered fully cached. When these evictions happen while prefetch is still going, we'll pass that check, as the the number of blocks for the file in the backingMap would still be 55, which is what we pass as dataBlockCount.

      As BucketCache is intended for storing data blocks only, I believe we should make sure BucketCache.notifyFileCachingCompleted only accounts for data blocks. Also, the TestBucketCachePersister.testPrefetchBlockEvictionWhilePrefetchRunning should be updated to consistently reproduce the eviction concurrent to the prefetch. 

       

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            wchevreuil Wellington Chevreuil Assign to me
            wchevreuil Wellington Chevreuil
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment