Details
-
Improvement
-
Status: In Progress
-
Minor
-
Resolution: Unresolved
-
3.3.0
-
None
-
None
Description
Map task will try to push all map outputs to external shuffle service now.
After the shuffle stage is finalized, the reduce fetch blocks RPC will be blocked if there are still many map output blocks in flight.
We could stop pushing the remaining blocks if the shuffle stage is finalized.