Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-914

(Umbrella) Support graceful decommission of nodemanager

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.0.4-alpha
    • None
    • graceful
    • None

    Description

      When NMs are decommissioned for non-fault reasons (capacity change etc.), it's desirable to minimize the impact to running applications.

      Currently if a NM is decommissioned, all running containers on the NM need to be rescheduled on other NMs. Further more, for finished map tasks, if their map output are not fetched by the reducers of the job, these map tasks will need to be rerun as well.

      We propose to introduce a mechanism to optionally gracefully decommission a node manager.

      Attachments

        Issue Links

          depends upon

          Sub-task - The sub-task of the issue YARN-311 Dynamic node resource configuration: core scheduler changes

          • Major - Major loss of function.
          • Closed
          duplicates

          Sub-task - The sub-task of the issue YARN-671 Add an interface on the RM to move NMs into a maintenance state

          • Major - Major loss of function.
          • Resolved
          is blocked by

          New Feature - A new feature of the product, which has yet to be developed. YARN-291 [Umbrella] Dynamic resource configuration

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-9160 Adopt Jolokia as the JMX HTTP/JSON bridge.

          • Major - Major loss of function.
          • Open
          is related to

          Sub-task - The sub-task of the issue YARN-671 Add an interface on the RM to move NMs into a maintenance state

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. YARN-4939 the decommissioning Node should keep alive if NM restart

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. YARN-1996 Provide alternative policies for UNHEALTHY nodes.

          • Major - Major loss of function.
          • Open
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-5381 Support graceful decommission of tasktracker

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              junping_du Junping Du
              vicaya Luke Lu
              Votes:
              3 Vote for this issue
              Watchers:
              74 Start watching this issue

              Dates

                Created:
                Updated: