Details
-
Bug
-
Status: Resolved
-
P1
-
Resolution: Fixed
-
None
-
None
Description
The Flink Runner's ImpulseSourceFunction does not emit a final watermark, unless --shutdownSourcesOnFinalWatermark flag has been specified (the flag is used in tests to shutdown the pipeline after reading all data). Most pipelines will be long-running and thus do not specify the flag.
Not sending out the final watermark causes GroupByKey to hold back the data of event time windows until the pipeline is shut down (the final watermark is always emitted on pipeline shutdown which is why using the above flag works).
Attachments
Issue Links
- links to