Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-14058 FLIP-53 Fine-grained Operator Resource Management
  3. FLINK-14060

Set slot sharing groups according to pipelined regions

    XMLWordPrintableJSON

Details

    Description

      StreamingJobGraphGenerator set slot sharing group for operators at compiling time.
      Identify pipelined regions, with respect to allSourcesInSamePipelinedRegion
      Set slot sharing groups according to pipelined regions 
      By default, each pipelined region should go into a separate slot sharing group
      If the user sets operators in multiple pipelined regions into same slot sharing group, it should be respected

      The slot sharing group of a vertex is determined as:

      • if a vertex slot sharing group is specified to be null, the slot sharing group is null
      • if a vertex slot sharing group is specified to be a non-default value(the group name not "default"), it will be respected
      • if a vertex slot sharing group is "default", it will be the slot sharing group of the vertex's logical pipelined region

      The slot sharing group of a region is determined as:

      • if allVerticesInSameSlotSharingGroupByDefault is true, all regions are in the same "default" slot sharing group
      • if allVerticesInSameSlotSharingGroupByDefault is false, all regions will be in distinct slot sharing groups

      This step should not introduce any behavior changes, given that later scheduled pipelined regions can reuse slots from previous scheduled pipelined regions. 

      Attachments

        Issue Links

          Activity

            People

              zhuzh Zhu Zhu
              xtsong Xintong Song
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m