Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-9580

Fulfilled reservation information in assignment is lost when transferring in ParentQueue#assignContainers

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.3.0, 3.2.1
    • capacityscheduler
    • None
    • Reviewed

    Description

      When transferring assignment from child queue to parent queue, fulfilled reservation information including fulfilledReservation and fulfilledReservedContainer in assignment is lost.

      When multi-nodes enabled, this lost can raise a problem that allocation proposal is generated but can't be accepted because there is a check for fulfilled reservation information in FiCaSchedulerApp#commonCheckContainerAllocation, this endless loop will always be there and the resource of the node can't be used anymore.

      In HB-driven scheduling mode, fulfilled reservation can be allocated via another calling stack: CapacityScheduler#allocateContainersToNode --> CapacityScheduler#allocateContainerOnSingleNode --> CapacityScheduler#allocateFromReservedContainer, in this way assignment can be generated by leaf queue and directly submitted, I think that's why we hardly find this problem before.

      Attachments

        1. YARN-9580.branch-3.2.002.patch
          6 kB
          Tao Yang
        2. YARN-9580.branch-3.2.001.patch
          5 kB
          Tao Yang
        3. YARN-9580.001.patch
          5 kB
          Tao Yang

        Activity

          People

            Tao Yang Tao Yang
            Tao Yang Tao Yang
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: