Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-27365

Minimise block addition failures due to no space in bucket cache writers queue by introducing wait time

    XMLWordPrintableJSON

Details

    Description

      Currently in bucket cache asynchronous caching mechanism introduced where initially the blocks to be cached will be added to queue and writer threads consume the blocks from the queue and write to bucket cache. In case if block writing to bucket cache is slow then there is a chance that queue of writer threads become full and following block additions will be failed. In case of slower storages like s3 might introduce latencies even if we enable bigger sizes of bucket cache using ephemeral storages. So we can allow configurable wait time while adding blocks to queue so that chances of queue free up is possible during the wait time and block addition failures can be minimised. To avoid the performance impact of wait time in regular read paths we can use the wait time mainly during background operations like compactions, flushes or prefetches etc.

      Attachments

        Issue Links

          Activity

            People

              rajeshbabu Rajeshbabu Chintaguntla
              rajeshbabu Rajeshbabu Chintaguntla
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: