Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6501

Improve reducer preemption

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.7.1
    • None
    • applicationmaster
    • None

    Description

      As discussed on MAPREDUCE-6302, we could improve the reducer preemption as follows:

      1. preempt enough reducers so there are enough mappers to match slowstart
      2. prioritize preempting reducers that are still in SHUFFLE phase
      3. add an option to not preempt reducers that are past SHUFFLE phase irrespective of slowstart as long as one mapper can run

      We could do it all in one patch or create subtasks as necessary.

      Attachments

        Issue Links

          Activity

            People

              pbacsko Peter Bacsko
              kasha Karthik Kambatla
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated: