Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-7394

Run local Mapreduce job failed when using nativetask

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Critical
    • Resolution: Unresolved
    • 3.1.1
    • None
    • mrv2, nativetask
    • None

    Description

      mapred-site.xml

       

      <configuration>
               <property>
                  <name>mapreduce.job.map.output.collector.class&lt;/name>
                  <value>org.apache.hadoop.mapred.nativetask.NativeMapOutputCollectorDelegator</value>
           </property>
      </configuration> 

      command:

       hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.1.jar pi 1 2 

      exception:

      java.lang.Exception: org.apache.hadoop.mapreduce.task.reduce.Shuffle$ShuffleError: error in shuffle in localfetcher#1
              at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:492)
              at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:559)
      Caused by: org.apache.hadoop.mapreduce.task.reduce.Shuffle$ShuffleError: error in shuffle in localfetcher#1
              at org.apache.hadoop.mapreduce.task.reduce.Shuffle.run(Shuffle.java:134)
              at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:377)
              at org.apache.hadoop.mapred.LocalJobRunner$Job$ReduceTaskRunnable.run(LocalJobRunner.java:347)
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
              at java.util.concurrent.FutureTask.run(FutureTask.java:266)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      Caused by: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find output/file.out in any of the configured local directories
              at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:494)
              at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:166)
              at org.apache.hadoop.mapred.MROutputFiles.getOutputFile(MROutputFiles.java:57)
              at org.apache.hadoop.mapreduce.task.reduce.LocalFetcher.copyMapOutput(LocalFetcher.java:120)
              at org.apache.hadoop.mapreduce.task.reduce.LocalFetcher.doCopy(LocalFetcher.java:103)
              at org.apache.hadoop.mapreduce.task.reduce.LocalFetcher.run(LocalFetcher.java:86)
       

       

      Attachments

        Activity

          People

            Unassigned Unassigned
            chenfengge chenfengge
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: