Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6608

Work Preserving AM Restart for MapReduce

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsAdd voteVotersWatch issueWatchersCreate sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      Providing a framework for work preserving AM is achieved in YARN-1489. We would like to take advantage of this for MapReduce(MR) applications. There are some challenges which have been described in the attached document and few options discussed. We solicit feedback from the community.

      Attachments

        1. Patch1.patch
          110 kB
          Srikanth Sampath
        2. WorkPreservingMRAppMaster.pdf
          109 kB
          Srikanth Sampath
        3. WorkPreservingMRAppMaster-1.pdf
          113 kB
          Srikanth Sampath
        4. WorkPreservingMRAppMaster-2.pdf
          126 kB
          Srikanth Sampath

        Issue Links

        depends upon

        Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-5485 Allow repeating job commit by extending OutputCommitter API

        • Critical - Crashes, loss of data, severe memory leak.
        • Closed

        Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-4815 Speed up FileOutputCommitter#commitJob for many output files

        • Major - Major loss of function.
        • Closed
        relates to

        Sub-task - The sub-task of the issue YARN-4602 Simple and Scalable Message Service for YARN application

        • Major - Major loss of function.
        • Open

        Bug - A problem which impairs or prevents the functions of the product. YARN-1489 [Umbrella] Work-preserving ApplicationMaster restart

        • Major - Major loss of function.
        • Resolved

        New Feature - A new feature of the product, which has yet to be developed. YARN-4758 Enable discovery of AMs by containers

        • Major - Major loss of function.
        • Open

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            srikanth.sampath Srikanth Sampath Assign to me
            srikanth.sampath Srikanth Sampath
            Votes:
            0 Vote for this issue
            Watchers:
            27 Start watching this issue

            Dates

              Created:
              Updated:

              Slack

                Issue deployment