Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2027

fair share scheduler causes unnecessary preemption under some conditions

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • contrib/fair-share
    • None

    Description

      When the cluster hits speculative cap - JobInProgress does not speculate any more. however FairShare Scheduler continues to believe that tasks need to be speculated.

      Under such conditions, FS Scheduler will start preempting repeatedly (because from it's perspective - certain jobs are not getting the slots they deserve (for speculative tasks)).

      There may be other reasons for excessive preemption as well - this one stood out clearly.

      Attachments

        Activity

          People

            Unassigned Unassigned
            jsensarma Joydeep Sen Sarma
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: