Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.9.0
    • 0.9.0
    • container, yarn
    • None

    Description

      SAMZA-348 has a detailed design proposal on how we can configure Samza via a stream. Part of this work involves converting the SamzaContainer to retrieve its information via an HTTP/JSON request, rather than from JSON encoded environment variables.

      The three items that we'll need to serve via a "job coordinator" will be the job's config, the job's container:task:SSP mappings, and the job's task:changelog partition mappings.

      We'll also need to assign each container a unique ID, so that it can retrieve its task:SSP mappings (and thus, its task:changelog partition mappings).

      This ticket does not encompass creating a "job coordinator". Instead, we'll just start the HTTP server in three places: the ThreadJob, the ProcessJob, and the YARN AM. In all three cases, a URL will be set via an environment variable, which the SamzaContainer will receive, and use to retrieve its information.

      Attachments

        1. SAMZA-438-3.patch
          91 kB
          Chris Riccomini
        2. SAMZA-438-2.patch
          91 kB
          Chris Riccomini
        3. SAMZA-438-1.patch
          89 kB
          Chris Riccomini
        4. SAMZA-438-0.patch
          88 kB
          Chris Riccomini

        Activity

          People

            criccomini Chris Riccomini
            criccomini Chris Riccomini
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: