Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-5713

Flink portable runner schedules all tasks of streaming job on same task manager

    XMLWordPrintableJSON

    Details

      Description

      The cluster has 9 task managers and 144 task slots total. A simple streaming pipeline with parallelism of 8 will get all tasks scheduled on the same task manager, causing the host to be fully booked and the remaining cluster idle.

        Attachments

        1. With RichParallelSourceFunction and parallelism 5.png
          5 kB
          Maximilian Michels
        2. Different SlotSharingGroup.png
          4 kB
          Maximilian Michels
        3. image-2018-10-11-11-43-50-333.png
          221 kB
          Thomas Weise
        4. image-2018-10-11-16-20-45-221.png
          176 kB
          Thomas Weise

          Issue Links

            Activity

              People

              • Assignee:
                mxm Maximilian Michels
                Reporter:
                thw Thomas Weise
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 20m
                  1h 20m