Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-1197

Support changing resources of an allocated container

    XMLWordPrintableJSON

Details

    Description

      The current YARN resource management logic assumes resource allocated to a container is fixed during the lifetime of it. When users want to change a resource
      of an allocated container the only way is releasing it and allocating a new container with expected size.
      Allowing run-time changing resources of an allocated container will give us better control of resource usage in application side

      Attachments

        1. YARN-1197_Design.2015.08.21.pdf
          692 kB
          MENG DING
        2. YARN-1197_Design.2015.07.07.pdf
          700 kB
          MENG DING
        3. YARN-1197_Design.2015.06.24.pdf
          685 kB
          MENG DING
        4. YARN-1197 old-design-docs-patches-for-reference.zip
          813 kB
          Wangda Tan
        5. YARN-1197_Design.pdf
          692 kB
          MENG DING

        Issue Links

          1.
          Common PB type definitions for container resizing Sub-task Closed Wangda Tan
          2.
          AM-RM protocol changes to support container resizing Sub-task Closed Wangda Tan
          3.
          AM-RM protocol changes to support container resizing Sub-task Resolved MENG DING
          4.
          AM-NM protocol changes to support container resizing Sub-task Resolved MENG DING
          5.
          ContainerManager implementation to support container resizing Sub-task Resolved MENG DING
          6.
          RM-NM protocol changes and NodeStatusUpdater implementation to support container resizing Sub-task Resolved MENG DING
          7.
          ContainerImpl changes to support container resizing Sub-task Resolved MENG DING
          8.
          ContainerManager recovery for container resizing Sub-task Resolved MENG DING
          9.
          Make ContainersMonitor can support change monitoring size of an allocated container in NM side Sub-task Resolved MENG DING
          10.
          CapacityScheduler side changes to support increase/decrease container resource. Sub-task Resolved Wangda Tan
          11.
          Support change container resource in RM Sub-task Resolved Wangda Tan
          12.
          Roll back container resource allocation after resource increase token expires Sub-task Resolved MENG DING
          13.
          Make AMRMClient support send increase container request and get increased/decreased containers Sub-task Resolved MENG DING
          14.
          Make NMClient support change container resources Sub-task Resolved MENG DING
          15.
          Add implementations to FairScheduler to support increase/decrease container resource Sub-task Patch Available Wilfred Spiegelenburg
          16.
          Protocol changes in RM side to support change container resource Sub-task Resolved Unassigned
          17.
          [YARN-1197] Add increased/decreased container to Allocation Sub-task Resolved Unassigned
          18.
          [YARN-1197] Modify ApplicationMasterService to support changing container resource Sub-task Resolved Unassigned
          19.
          [YARN-1197] Modify ResourceTrackerService to support passing decreased containers to RMNode Sub-task Resolved Unassigned
          20.
          [YARN-1197] Add pullDecreasedContainer API to RMNode which can be used by scheduler to get newly decreased Containers Sub-task Resolved Unassigned
          21.
          [YARN-1197] Add methods in FiCaSchedulerNode to support increase/decrease/reserve/unreserve change container requests/results Sub-task Resolved Unassigned
          22.
          [YARN-1197] Add APIs in CSQueue to support decrease container resource and unreserve increase request Sub-task Resolved Unassigned
          23.
          [YARN-1197] Add implementations to CapacityScheduler to support increase/decrease container resource Sub-task Resolved Unassigned
          24.
          Merge YARN-1197 back to trunk Sub-task Resolved Wangda Tan
          25.
          Support changing container cpu resource Sub-task Resolved Yang Wang
          26.
          Resolve findbugs/javac warnings in YARN-1197 branch Sub-task Resolved Wangda Tan
          27.
          Example of use YARN-1197 Sub-task In Progress MENG DING
          28.
          Increasing container resource while there is no headroom left will cause ResourceManager to crash Sub-task Resolved MENG DING

          Activity

            People

              Unassigned Unassigned
              leftnoteasy Wangda Tan
              Votes:
              13 Vote for this issue
              Watchers:
              119 Start watching this issue

              Dates

                Created:
                Updated: