[BEAM-13142] Ensure that the payload is consumed on Elements.Data and Elements.Timers when is_last = true - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Open
Priority: P3
Resolution: Unresolved
Affects Version/s: None
Fix Version/s: None
Component/s: sdk-go, sdk-java-harness, sdk-py-harness
Labels:
None

Description

BeamFnDataInboundObserver drops the bytes instead of passing it forward on the last message: https://github.com/apache/beam/blob/cd4b7f3b1af4f51bdab1a0b1a98f94b5288c09ec/sdks/java/fn-execution/src/main/java/org/apache/beam/sdk/fn/data/BeamFnDataInboundObserver.java#L58

data_plane.py drops the timers payload but seems to do the right thing for data:
https://github.com/apache/beam/blob/cd4b7f3b1af4f51bdab1a0b1a98f94b5288c09ec/sdks/python/apache_beam/runners/worker/data_plane.py#L343

data_mgr.go drops the data payload (timers don't seem to be supported):
https://github.com/apache/beam/blob/cd4b7f3b1af4f51bdab1a0b1a98f94b5288c09ec/sdks/go/pkg/beam/core/runtime/harness/datamgr.go#L297

Attachments

Sub-Tasks

1.

[Go SDK] Ensure that the payload is consumed on Elements.Data is_last = true

Resolved

Robert Burke

100%

Original Estimate - Not Specified

Original Estimate - Not Specified

Time Spent - 50m

Activity

People

Assignee:: Unassigned

Reporter:: Luke Cwik

Votes:: 0 Vote for this issue

Watchers:: 5 Start watching this issue

Dates

Created:: 28/Oct/21 21:00

Updated:: 04/Jun/22 21:45

Time Tracking

Estimated:

Not Specified

Remaining:

0h

Logged:

50m

Include sub-tasks