[CASSANDRA-15869] MemoryOutputStream overflow on large bloom filter - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Normal
Resolution: Fixed
Fix Version/s: 4.0-beta1, 4.0
Component/s: Local/SSTable
Labels:
None

Bug Category:
Correctness - Recoverable Corruption / Loss
Severity:
Normal
Complexity:
Normal
Discovered By:
Performance Regression Test
Platform:

All
Impacts:

None
Since Version:

4.0-alpha1
Source Control Link:

https://github.com/apache/cassandra/commit/7cdad3ce61664cf2ffac75b6e537c910e573754c
Test and Documentation Plan:

Hide

Large sstable can be loaded again.

CircleCi: https://circleci.com/workflow-run/69685b9a-f356-46ef-939f-851cc1d29a4f

Show
Large sstable can be loaded again. CircleCi: https://circleci.com/workflow-run/69685b9a-f356-46ef-939f-851cc1d29a4f

Description

Since ~~CASSANDRA-9067~~, Cassandra will use MemoryOutputStream to reconstruct BF Memory without re-ordering bytes, see OffsetBitSet#deserialize. But MemoryOutputStream use INT to track position and will overflow when BF size exceeds 2GB.

stacktrace

error: Illegal bounds [-2147483648..-2147483584); size: 4588588016
-- StackTrace --
java.lang.AssertionError: Illegal bounds [-2147483648..-2147483584); size: 4588588016
	at org.apache.cassandra.io.util.Memory.checkBounds(Memory.java:185)
	at org.apache.cassandra.io.util.Memory.setBytes(Memory.java:138)
	at org.apache.cassandra.io.util.MemoryOutputStream.write(MemoryOutputStream.java:45)

Attachments

Issue Links

is related to

CASSANDRA-9067 BloomFilter serialization format should not change byte ordering

Resolved

Activity

People

Assignee:: Zhao Yang

Reporter:: Zhao Yang

Authors:: Zhao Yang

Reviewers:: Brandon Williams

Votes:: 0 Vote for this issue

Watchers:: 2 Start watching this issue

Dates

Created:: 10/Jun/20 10:16

Updated:: 03/Jan/21 17:03

Resolved:: 12/Jun/20 14:17

Time Tracking

Estimated:

Not Specified

Remaining:

Logged:

20m