Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6912

CompletedMapTask contains failed and killed will lead mapreduce.job.reduce.slowstart.completedmaps not work properly

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.6.2
    • Fix Version/s: None
    • Component/s: mrv2
    • Labels:
      None

      Description

      if the conf is set to 1, is about to make reduce task to be scheduled only after all map task success

      int completedMapsForReduceSlowstart = (int)Math.ceil(reduceSlowStart *
      totalMaps);
      if(completedMaps < completedMapsForReduceSlowstart) {}
      
      ///////////////////////////
      
      int completedMaps = getJob().getCompletedMaps();
      
      ///////////////////////////
      JobImpl
       @Override
      public int getCompletedMaps() {
      readLock.lock();
      try {
      return succeededMapTaskCount + failedMapTaskCount + killedMapTaskCount;
      } finally {
      readLock.unlock();
      }
      }
      

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              SuYan SuYan
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: