Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-73

Add topologies to Samza

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Wish
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 0.6.0
    • None
    • build, container, yarn

    Description

      This is an exploratory ticket to figure out if we want to support topologies in Samza.

      Potential solutions include:

      1. Don't support them as part of the framework (a la Hadoop).
      2. Use Tez as topology runner.
      3. Use config inheritence (.properties file merging). This already exists.
      4. Write some kind of JobGroupRunner that executes multiple jobs at one time.
      5. ?

      There are probably a lot of solutions. Some goals of this ticket are:

      a. Config sharing between job with a lot of the same config.
      b. Whether we want to run multiple jobs with a single AM (in YARN).
      c. How to start/stop only a subset of job when running in a topology.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            criccomini Chris Riccomini

            Dates

              Created:
              Updated:

              Slack

                Issue deployment