[HBASE-15477] Do not save 'next block header' when we cache hfileblocks - ASF JIRA

Voters

Stop watching

Watchers

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Sub-task
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 1.3.0, 2.0.0
Component/s: BlockCache, Performance
Labels:
None

Hadoop Flags:

Reviewed
Release Note:

Hide
Fix over-persisting in blockcache; no longer save the block PLUS the header of the next block (33 bytes) when writing the cache.

Also removes support for hfileblock v1; hfile block v1 was used writing hfile v1. hfile v1 was the default in hbase before hbase-0.92. hbase.96 would not start unless all v1 hfiles had been compacted out of the cluster.

Show
Fix over-persisting in blockcache; no longer save the block PLUS the header of the next block (33 bytes) when writing the cache. Also removes support for hfileblock v1; hfile block v1 was used writing hfile v1. hfile v1 was the default in hbase before hbase-0.92. hbase.96 would not start unless all v1 hfiles had been compacted out of the cluster.

Description

When we read from HDFS, we overread to pick up the next blocks header.
Doing this saves a seek as we move through the hfile; we save having to
do an explicit seek just to read the block header every time we need to
read the body. We used to read in the next header as part of the
current blocks buffer. This buffer was then what got persisted to
blockcache; so we were over-persisting wrtiting out our block plus the
next blocks' header (overpersisting 33 bytes). Parse of HFileBlock
complicated by this extra tail. Fix.