Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
Impala 4.3.0
-
None
-
None
-
ghx-label-7
Description
MERGING-EXCHANGE dedicate one row batch queue per data stream sender with soft limit of 10MB per queue. For large cluster with hundreds of backend executor, this can lead MERGING-EXCHANGE to require hundreds of queues totaling hundreds MBs of memory.
We need to improve the design to keep memory of coordinator fragment low.