[BEAM-9014] Update CachingShuffleBatchReader to record weights by size in bytes - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Resolved
Priority: P3
Resolution: Incomplete
Affects Version/s: None
Fix Version/s: 2.21.0
Component/s: runner-dataflow
Labels:
None

Description

Currently the CachingShuffleBatchReader caches based upon the number of batches and not the size of those batches. This task is about updating CachingShuffleBatchReader to cache based on the size of those batches.

Attachments

Issue Links

links to

GitHub Pull Request #10418

Activity

People

Assignee:: Tyson Hamilton

Reporter:: Luke Cwik

Votes:: 0 Vote for this issue

Watchers:: 2 Start watching this issue

Dates

Created:: 20/Dec/19 23:57

Updated:: 24/Jul/20 20:00

Resolved:: 21/Apr/20 22:08

Time Tracking

Estimated:

Not Specified

Remaining:

Logged:

1.5h