Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4399

FairScheduler allocated container should resetSchedulingOpportunities count of its priority

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • 2.7.1
    • None
    • fairscheduler

    Description

      There is a bug on fairScheduler allocating containers when you configurate the locality configs.When you attempt to assigned a container,it will invoke FSAppAttempt#addSchedulingOpportunity whenever it can be assigned successfully or not. And if you configurate the yarn.scheduler.fair.locality.threshold.node and yarn.scheduler.fair.locality.threshold.rack, the schedulingOpportunity value will influence the locality of containers.Because if one container is assigned successfully and its priority schedulingOpportunity count will be increased, and second container will be increased again.This will may be let their priority of allowedLocality degrade. And this will let this container dealt by rackRequest. So I think in fairScheduler allocating container, if the previous container was dealt, its priority of schedulerCount should be reset to 0, and don't let its value influence container's allocating in next iteration and this will increased the locality of containers.

      Attachments

        1. YARN-4399.003.patch
          8 kB
          Yiqun Lin
        2. YARN-4399.002.patch
          7 kB
          Yiqun Lin
        3. YARN-4399.001.patch
          7 kB
          Yiqun Lin

        Issue Links

          Activity

            People

              linyiqun Yiqun Lin
              linyiqun Yiqun Lin
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: