[NEMO-350] Implement Off-heap SerializedMemoryStore - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Sub-task
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 0.2
Labels:
- gsoc2019
- pull-request-available

Description

Currently, Nemo SerializedMemoryStore stores its data inside JVM heap, which may cause excessive garbage collection overhead and makes the overall job completion time longer.

One way to alleviate this problem is moving data to on-heap region to off-heap region and manages memory allocation by ourselves. To implement this feature, we need to use specialized features such as `ByteBuffer` or `Unsafe`.

Attachments

Issue Links

incorporates

NEMO-388 Off-heap memory management (reuse ByteBuffer)

Resolved

NEMO-383 Implement DirectByteBufferOutputStream for Off-heap SerializedMemoryStore

Resolved

NEMO-384 Implement DirectByteBufferInputStream for Off-heap SerializedMemoryStore

Resolved

links to

GitHub Pull Request #221

GitHub Pull Request #222

Activity

People

Assignee:: Haeyoon Cho

Reporter:: Gyewon Lee

Votes:: 0 Vote for this issue

Watchers:: 2 Start watching this issue

Dates

Created:: 07/Mar/19 02:33

Updated:: 07/Mar/20 06:36

Resolved:: 25/Jun/19 06:30

Time Tracking

Estimated:

Not Specified

Remaining:

Logged:

40m