Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4339

pi example job hangs on when run on hadoop 0.23.0 when capacity scheduler is included in the setting environment.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Cannot Reproduce
    • 0.23.0
    • 0.23.0
    • Ubuntu Server 11.04, Hadoop 0.23.0,

    • Hadoop

    Description

      Tried to include default capacity scheduler in hadoop and tried to run an example pi program. The job hangs and no more output is getting displayed.
      Starting Job
      2012-06-12 22:10:02,524 INFO ipc.YarnRPC (YarnRPC.java:create(47)) - Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC
      2012-06-12 22:10:02,538 INFO mapred.ResourceMgrDelegate (ResourceMgrDelegate.java:<init>(95)) - Connecting to ResourceManager at localhost/127.0.0.1:8030
      2012-06-12 22:10:02,539 INFO ipc.HadoopYarnRPC (HadoopYarnProtoRPC.java:getProxy(48)) - Creating a HadoopYarnProtoRpc proxy for protocol interface org.apache.hadoop.yarn.api.ClientRMProtocol
      2012-06-12 22:10:02,665 INFO mapred.ResourceMgrDelegate (ResourceMgrDelegate.java:<init>(99)) - Connected to ResourceManager at localhost/127.0.0.1:8030
      2012-06-12 22:10:02,727 WARN conf.Configuration (Configuration.java:handleDeprecation(326)) - fs.default.name is deprecated. Instead, use fs.defaultFS
      2012-06-12 22:10:02,728 WARN conf.Configuration (Configuration.java:handleDeprecation(343)) - mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used
      2012-06-12 22:10:02,831 INFO input.FileInputFormat (FileInputFormat.java:listStatus(245)) - Total input paths to process : 10
      2012-06-12 22:10:02,900 INFO mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(362)) - number of splits:10
      2012-06-12 22:10:03,044 INFO mapred.YARNRunner (YARNRunner.java:createApplicationSubmissionContext(279)) - AppMaster capability = memory: 2048
      2012-06-12 22:10:03,286 INFO mapred.YARNRunner (YARNRunner.java:createApplicationSubmissionContext(355)) - Command to launch container for ApplicationMaster is : $JAVA_HOME/bin/java -Dlog4j.configuration=container-log4j.properties -Dyarn.app.mapreduce.container.log.dir=<LOG_DIR> -Dyarn.app.mapreduce.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Xmx1536m org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1><LOG_DIR>/stdout 2><LOG_DIR>/stderr
      2012-06-12 22:10:03,370 INFO mapred.ResourceMgrDelegate (ResourceMgrDelegate.java:submitApplication(304)) - Submitted application application_1339507608976_0002 to ResourceManager
      2012-06-12 22:10:03,432 INFO mapreduce.Job (Job.java:monitorAndPrintJob(1207)) - Running job: job_1339507608976_0002
      2012-06-12 22:10:04,443 INFO mapreduce.Job (Job.java:monitorAndPrintJob(1227)) - map 0% reduce 0%

      Attachments

        Activity

          People

            Unassigned Unassigned
            srikraj8341 srikanth ayalasomayajulu
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - 48h
                48h
                Remaining:
                Remaining Estimate - 48h
                48h
                Logged:
                Time Spent - Not Specified
                Not Specified