[BEAM-7912] Optimize GroupIntoBatches for batch Dataflow pipelines - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Open
Priority: P3
Resolution: Unresolved
Affects Version/s: None
Fix Version/s: None
Component/s: runner-dataflow
Labels:
None

Description

The GroupIntoBatches transform can be significantly optimized on Dataflow since it always ensures that a key K appears in only one bundle after a GroupByKey. This removes the usage of state and timers in the generic GroupIntoBatches transform.

Attachments

Issue Links

links to

GitHub Pull Request #9280

Activity

People

Assignee:: Unassigned

Reporter:: Luke Cwik

Votes:: 0 Vote for this issue

Watchers:: 2 Start watching this issue

Dates

Created:: 06/Aug/19 20:09

Updated:: 04/Jun/22 13:01

Time Tracking

Estimated:

Not Specified

Remaining:

Logged:

1h 50m