Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-14058 FLIP-53 Fine-grained Operator Resource Management
  3. FLINK-14974

Managed memory fractions should be rounded properly to not summed up to be more than 1.0

    XMLWordPrintableJSON

Details

    Description

      Managed memory fractions should be rounded to floor at a certain precision when divided by the number of operators, otherwise the fractions can be summed up to be more than 1.0 due to the double precision issue, and the last operator may fail to allocate managed memory it is supposed to be able to acquire.

      To achieve that, I think we should change StreamingJobGraphGenerator#setManagedMemoryFractionForOperator to use BigDecimal#divide(otherValue, scale, RoundMode.ROUND_DOWN) to calculate the fractions. In this way, the sum of the fractions will not exceed 1.0.
      The scale can be a bit larger (maybe 16) so that we only lose little managed memory.

      cc chesnay

      Attachments

        Issue Links

          Activity

            People

              zhuzh Zhu Zhu
              zhuzh Zhu Zhu
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m