Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6892

Issues with the count of failed/killed tasks in the jhist file

    XMLWordPrintableJSON

Details

    • Incompatible change, Reviewed
    • This adds some new job counters, the number of failed MAP/REDUCE tasks and the number of killed MAP/REDUCE tasks.

    Description

      Recently we encountered some issues with the value of failed tasks. After parsing the jhist file, JobInfo.getFailedMaps() returned 0, but actually there were failures.

      Another minor thing is that you cannot get the number of killed tasks (although this can be calculated).

      The root cause is that JobUnsuccessfulCompletionEvent contains only the successful map/reduce task counts. Number of failed (or killed) tasks are not stored.

      Attachments

        1. MAPREDUCE-6892-006.patch
          70 kB
          Peter Bacsko
        2. MAPREDUCE-6892-005.patch
          70 kB
          Peter Bacsko
        3. MAPREDUCE-6892-004.patch
          68 kB
          Peter Bacsko
        4. MAPREDUCE-6892-003.patch
          67 kB
          Peter Bacsko
        5. MAPREDUCE-6892-002.PATCH
          38 kB
          Peter Bacsko
        6. MAPREDUCE-6892-001.patch
          30 kB
          Peter Bacsko

        Activity

          People

            pbacsko Peter Bacsko
            pbacsko Peter Bacsko
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: