Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-14058 FLIP-53 Fine-grained Operator Resource Management
  3. FLINK-14536

Make clear the way to aggregate specified cpuCores resources

    XMLWordPrintableJSON

Details

    Description

      I'm raising this question because I find cpuCores in ResourceSpec#merge are aggregated with max(), while in ResourceProfile#merge it is sum().

      This means that when calculating resources of a vertex from within operators, the cpuCores is the max value. While it is a sum(or subtraction in ResourceProfile#subtract) when dealing with shared slot bookkeeping and related checks.
      This is confusing to me, especially when I'm considering how to generate a shared slot resource spec merged from all vertices in it(see FLINK-14314).

      I'm not pretty sure if we already have a concise definition for it?
      If there is, we need to respect it and change ResourceSpec or ResourceProfile accordingly.
      If not, we need to decide it first before we can move on with fine grained resources.

      Need to mention that if we take the max() way, we need to re-consider how we can conduct a correct ResourceProfile#subtract regarding cpuCores, since there is not a clear way to reverse a max() computation.

      cc trohrmann xintongsong azagrebin

      Attachments

        Issue Links

          Activity

            People

              zhuzh Zhu Zhu
              zhuzh Zhu Zhu
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m