Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-9359

Avoid code duplication in Resources for calculation methods

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: In Progress
    • Minor
    • Resolution: Unresolved
    • None
    • None
    • None

    Description

      This is a follow-up for YARN-9318, dealing with code duplication issueas, as discussed with templedf earlier.

      Resources has many very similar calculation methods like addTo, subtractFrom, multiply, etc.
      These are having extractable code as common, the only difference could be the calculation they perform on the passed Resource object(s).

      These methods either receive one or two Resource objects and make some calculations on these.
      One caveat that needs some attention is that some of them do clone the Resource and do the calculation on the cloned resource and return the result (leaving the passed Resource alone) and some of them perform the calculation on the passed Resource object itself.

      The common code could be extracted like this:

      private static Resource applyFunctionOnValues(Resource lhs,
            Function<Long, Long> valueFunction) {
          int numResources = ResourceUtils.getNumberOfCountableResourceTypes();
          for (int i = 0; i < numResources; i++) {
            try {
              ResourceInformation lhsValue = lhs.getResourceInformation(i);
              Long modifiedValue = valueFunction.apply(lhsValue.getValue());
              lhs.setResourceValue(i, modifiedValue);
            } catch (ResourceNotFoundException ye) {
              LOG.warn("Resource is missing:" + ye.getMessage());
            }
          }
          return lhs;
        }
      

      And an example code could be like this:

      public static Resource multiplyAndRoundUp(Resource lhs, double by) {
          return applyFunctionOnValues(clone(lhs),
              (value) -> (long) Math.ceil(value * by));
        }
      

      Attachments

        Activity

          People

            smileLee WEI-HSIAO-LEE
            snemeth Szilard Nemeth
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: