[HBASE-26274] Create an option to reintroduce BlockCache to mapreduce job - ASF JIRA

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: 3.0.0-alpha-1, 2.4.6
Fix Version/s: 3.0.0-alpha-1, 2.5.0, 2.3.7, 2.4.7
Component/s: BlockCache, HFile, mapreduce
Labels:
None

Release Note:

Hide
Introduce `hfile.onheap.block.cache.fixed.size` and default to disable. When using ClientSideRegionScanner, it will be enabled with a fixed size for caching INDEX/LEAF_INDEX block when a client, e.g. snapshot scanner, scans the entire HFile and does not need to seek/reseek to index block multiple times.

Show
Introduce `hfile.onheap.block.cache.fixed.size` and default to disable. When using ClientSideRegionScanner, it will be enabled with a fixed size for caching INDEX/LEAF_INDEX block when a client, e.g. snapshot scanner, scans the entire HFile and does not need to seek/reseek to index block multiple times.

Description

In ~~HBASE-21498~~ (see this commit, it change the behavior that only region server can initialize on-heap BlockCache/LruBlockCache, this should be the right change for HMaster.

Other downstream dependency that uses getScanner from a file-based region and read HStore/HFile lost the BlockCache for caching INDEX/LEAF_INDEX (at least still a problem with HBase-2.4) after this change (it worked before) and caused performance impact with 2x slower.

One way to bring back the performance is to allow non-RS and non-HMaster can use a compact version of blockcache with smaller memory and less hbase internal configuration.

Or if we can find a way to cache or skip reading the same LEAF_INDEX when scanning the DATA block with HFile.