Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-7291

Attempt(map or reduce) remains NEW(state). And job is stuck in certain conditions.

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsAdd voteVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.1.2
    • None
    • mrv2
    • None

    Description

      Attempt(map or reduce) remains NEW(state). And job is stuck in certain conditions.

      The following are the situations:

      1. total task(map/reduce) count is same as the running limit of task(mapreduce.job.running.map.limit/mapreduce.job.running.reduce.limit).
      2. And start job. -> And total tasks(map/reduce) are running. -> And failed attempt for some reasons.
      3. Request allocation of new containers because the attempt failed.
      4. Quickly receive allocation of new containers.
      5. However, new container is released because failed attempts have not been cleared up.(allocated == total == running limit)
      6. Subsequently, the failed attempts is terminated, but it is waiting forever.
      7. Job is stuck.

      We switched MR frameworks(2.7.1) and checked that it worked well.
      Perhaps it is related to MAPREDUCE-6697

      Can you help me?

       

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned Assign to me
            dasom Lee young gon

            Dates

              Created:
              Updated:

              Slack

                Issue deployment