Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-73

Add topologies to Samza

    XMLWordPrintableJSON

Details

    • Wish
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 0.6.0
    • None
    • build, container, yarn

    Description

      This is an exploratory ticket to figure out if we want to support topologies in Samza.

      Potential solutions include:

      1. Don't support them as part of the framework (a la Hadoop).
      2. Use Tez as topology runner.
      3. Use config inheritence (.properties file merging). This already exists.
      4. Write some kind of JobGroupRunner that executes multiple jobs at one time.
      5. ?

      There are probably a lot of solutions. Some goals of this ticket are:

      a. Config sharing between job with a lot of the same config.
      b. Whether we want to run multiple jobs with a single AM (in YARN).
      c. How to start/stop only a subset of job when running in a topology.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              criccomini Chris Riccomini
              Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: