Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-9173

FairShare calculation broken for large values after YARN-8833

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.3.0
    • 3.0.4, 3.1.2, 3.3.0, 3.2.1
    • fairscheduler
    • None
    • Reviewed

    Description

      After the fix for the infinite loop in YARN-8833 we now get the wrong values back for fairshare calculations under certain circumstances. The current implementation works when the total resource is smaller than Integer.MAXVALUE.

      When the total resource goes above that value the number of iterations is not enough to converge to the correct value.

      The new test testResourceUsedWithWeightToResourceRatio() only checks that the calculation does not hang but does not check the outcome of the calculation.

      Attachments

        1. YARN-9137-branch-3.1.001.patch
          14 kB
          Weiwei Yang
        2. YARN-9137-branch3.1.001.patch
          14 kB
          wilfreds#1
        3. YARN-9173.002.patch
          15 kB
          wilfreds#1
        4. YARN-9173.001.patch
          14 kB
          wilfreds#1

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            wilfreds Wilfred Spiegelenburg
            wilfreds Wilfred Spiegelenburg
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment