Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3135

There should be an easier way to submit Java-based YARN applications

    XMLWordPrintableJSON

Details

    Description

      Since YARN is designed to not only work with Java, it is quite difficult and cumbersome to correctly setup a YARN application. For Java-based ones, it should be easier; there should be something like the MR2 Job API. My proposal is the following:

      1. Have a basic Job / Application object, e.g. YarnApplication, which could be instantiated and configured the following:
        final Configuration conf = ...
        final YarnApplication app = YarnApplication.getInstance(conf);
        app.setContainerClassName(MyContainer.class);
        app.setJarByClass(MyContainer.class);
         
      2. There should be a means to send and receive messages between container and AppMaster
      3. The container should be able to report progress and state to the AppMaster
      4. Bonus: A simplified WebAppMaster should start a web application and have its URL automatically proxied when run

      Attachments

        Activity

          People

            Unassigned Unassigned
            rabejens Jens Rabe
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:

              Time Tracking

                Estimated:
                Original Estimate - 672h
                672h
                Remaining:
                Remaining Estimate - 672h
                672h
                Logged:
                Time Spent - Not Specified
                Not Specified