Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-6640

Reshuffle not translated to Flink rebalance

    XMLWordPrintableJSON

Details

    Description

      Flink runner as of 2.10 uses native rebalance to implement reshuffle, this is broken in master and leads to performance degradation.

      Attachments

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. BEAM-6243 TFX pipelines experience a huge blowup in intermediate data size

          • P3 - Non-urgent bugs, features, and improvements
          • Open

          New Feature - A new feature of the product, which has yet to be developed. BEAM-6294 Use Flink's redistribute for reshuffle.

          • P2 - Default priority. Will be triaged and planned according to community practices.
          • Resolved
          links to

          Web Link GitHub Pull Request #7798

          Activity

            People

              thw Thomas Weise
              thw Thomas Weise
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 40m
                  40m