Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-35166

Improve the performance of Hybrid Shuffle when enable memory decoupling

    XMLWordPrintableJSON

Details

    Description

      Currently, the tiered result partition creates the SortBufferAccumulator with the number of expected buffers as min(numSubpartitions+1, 512), thus the SortBufferAccumulator may obtain very few buffers when the parallelism is small. We can easily make the number of expected buffers 512 by default to have a better performance when the buffers are sufficient.

      Attachments

        Issue Links

          Activity

            People

              Jiang Xin Jiang Xin
              Jiang Xin Jiang Xin
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: