Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-28303

Interrupt cache prefetch thread when a heap usage threshold is reached

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.6.0, 2.4.17, 3.0.0-beta-1, 4.0.0-alpha-1, 2.5.7, 2.7.0
    • None
    • None
    • Hide
      This change adds a new configuration property, "hbase.rs.prefetchheapusage", which specifies a heap usage limit for the prefetch threads to be allowed to execute. This is to prevent an overhead heap usage by bucket cache mapping objects when using file or offheap engines.

      This value should be defined as a double representing the heap usage percentage, and it's default set to 1d (100% heap), disabling this feature. For example, to set an 80% heap usage threshold, set "hbase.rs.prefetchheapusage" to "0.8" in the RegionServer hbase-site.xml configuration.
      Show
      This change adds a new configuration property, "hbase.rs.prefetchheapusage", which specifies a heap usage limit for the prefetch threads to be allowed to execute. This is to prevent an overhead heap usage by bucket cache mapping objects when using file or offheap engines. This value should be defined as a double representing the heap usage percentage, and it's default set to 1d (100% heap), disabling this feature. For example, to set an 80% heap usage threshold, set "hbase.rs.prefetchheapusage" to "0.8" in the RegionServer hbase-site.xml configuration.

    Description

      Mostly critical when using non heap cache implementations, such as offheap or file based. If the cache medium is too large and there are many blocks to be cached, it may create a lot of cache index object in the RegionServer heap. We should have guardrails to preventing caching from exhausting available heap.

      Attachments

        Issue Links

          Activity

            People

              wchevreuil Wellington Chevreuil
              wchevreuil Wellington Chevreuil
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: