Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-1067

Support Amazon EMR action executor in oozie installed on EC2

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: trunk
    • Fix Version/s: None
    • Component/s: action, coordinator, workflow
    • Labels:
    • Environment:

      Oozie, Amazon EMR availability, EC2 instance, access to Amazon S3 or S3N filesystem.

      Description

      Oozie is being adopted as default workflow/scheduling engine for BigData.

      Currently, small organizations prefer on demand clusters like Amazon's EMR instead of full fledged Hadoop setup. However, currently we don't have support for powerful workflow engine like oozie, which seamlessly schedules/executes user jobs on EMR.

      Oozie can provide a new ActionExecutor class like EMRActionExecutor, which can take all the required credentials for EMR.
      Oozie can be installed on Amazon EC2 instance, which can then talk to any dynamic EMR cluster.
      Though, Oozie has support for other filesystems other than HDFS, we might need to tweak a bit to support Filesystems like S3.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              shaik.idris Shaik Idris Ali
            • Votes:
              1 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:

                Time Tracking

                Estimated:
                Original Estimate - 506h
                506h
                Remaining:
                Remaining Estimate - 506h
                506h
                Logged:
                Time Spent - Not Specified
                Not Specified