Details
-
Bug
-
Status: Resolved
-
P2
-
Resolution: Fixed
-
None
-
None
Description
The retried work is silently skipped in https://github.com/apache/beam/blob/f7cbf88f550c8918b99a13af4182d6efa07cd2b5/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingDataflowWorker.java#L2010
This result into skipping of worker release work the corresponding duplicate work item and causes result into incorrect worker allocation for new work items.
This does not impact the correctness of the job but it can have an impact on a long running streaming job with multiple SDKHarness like in case of python streaming.
Attachments
Issue Links
- links to