Details
-
Improvement
-
Status: Resolved
-
P3
-
Resolution: Fixed
-
None
-
None
Description
For streaming pipelines, we want to be able to lift the combiner into the MergeBuckets without having to also do a PartialGroupByKey before the shuffle. We don't want to do the PGBK since it could cause non-deterministic results when used with some triggers.
We propose adding a new URN for doing just the convert to accumulators step and adding support for it in Java/Python/Go.
Attachments
Issue Links
- links to