Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-14058 FLIP-53 Fine-grained Operator Resource Management
  3. FLINK-14314

Allocate shared slot resources respecting the resources of all vertices in the group

    XMLWordPrintableJSON

Details

    Description

      With FLINK-14058, it is assumed that a shared slot should be large enough to be used by one instance of each JobVertex in the group simultaneously.

      To support it, a shared slot resources should be the sum of all JobVertex resources in the group.

      Here's the concrete proposal:
      1. Add a physicalSlotResourceProfile in SlotProfile. If the task is not in a shared slot, it is the task resource profile. Otherwise it is the slot sharing group resource profile. It should be used for physical slot allocation. Rename previous ResourceProfile to be taskResourceProfile for logical slot allocation.
      2. SharedSlotOversubscribedException and its handling can be removed, including part of the children slots releasing and re-allocating. This is because partial fulfillment should not happen anymore with #1. A simple sanity check can be kept for oversubscribing.

      Attachments

        Issue Links

          Activity

            People

              zhuzh Zhu Zhu
              zhuzh Zhu Zhu
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m