Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-7291

Attempt(map or reduce) remains NEW(state). And job is stuck in certain conditions.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.1.2
    • None
    • mrv2
    • None

    Description

      Attempt(map or reduce) remains NEW(state). And job is stuck in certain conditions.

      The following are the situations:

      1. total task(map/reduce) count is same as the running limit of task(mapreduce.job.running.map.limit/mapreduce.job.running.reduce.limit).
      2. And start job. -> And total tasks(map/reduce) are running. -> And failed attempt for some reasons.
      3. Request allocation of new containers because the attempt failed.
      4. Quickly receive allocation of new containers.
      5. However, new container is released because failed attempts have not been cleared up.(allocated == total == running limit)
      6. Subsequently, the failed attempts is terminated, but it is waiting forever.
      7. Job is stuck.

      We switched MR frameworks(2.7.1) and checked that it worked well.
      Perhaps it is related to MAPREDUCE-6697

      Can you help me?

       

      Attachments

        1. image-2020-08-10-15-13-03-930.png
          226 kB
          Lee young gon

        Activity

          People

            Unassigned Unassigned
            dasom Lee young gon
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: