Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-7222

Map tasks' outputs can not be recovered when ApplicationMaster relaunched

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.7.3
    • Fix Version/s: None
    • Component/s: mrv2
    • Labels:
      None

      Description

      When AM crashes, Yarn would launch a new AM instance and recover all its scheduled tasks. However mapper tasks's committed output files are not recovered when the number of reducers > 0. In my application which output files from mapper and make use of reducer to collect statistics not able to fully recover from the AM crash, and resulting in data from the previous completed mapper tasks get lost in the final output dir.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              lfyangcoder yanglongfei
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: