Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-11957

Worker endpoints are unset in StartWorkerRequest.

Details

    • Bug
    • Status: Open
    • P3
    • Resolution: Unresolved
    • None
    • None
    • java-fn-execution
    • None

    Description

      User reported that worker endpoints are unset when attempting to start a worker in a worker pool using Kubernetes. https://stackoverflow.com/questions/66498209/how-to-configure-beam-python-sdk-with-spark-in-a-kubernetes-environment

      Starting worker with command ['/opt/apache/beam/boot', '--id=1-1', '--logging_endpoint=', '--artifact_endpoint=', '--provision_endpoint=', '--control_endpoint=']

      These endpoints are expected to be set to use localhost and some random allocated port. The server is started here: https://github.com/apache/beam/blob/b101064f18e71adb81c8fe6ba7beae53e9b13b08/runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/control/DefaultJobBundleFactory.java#L672-L697
      And the StartWorkerRequest is constructed here: https://github.com/apache/beam/blob/master/runners/java-fn-execution/src/main/java/org/apache/beam/runners/fnexecution/environment/ExternalEnvironmentFactory.java

      Attachments

        Activity

          People

            Unassigned Unassigned
            ibzib Kyle Weaver
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: