Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-7933

Adding timeout to JobServer grpc calls

Details

    Description

      grpc calls to the JobServer from the Python SDK do not have timeouts. That means that the call to pipeline.run()could hang forever if the JobServer is not running (or failing to start).
      E.g. https://github.com/apache/beam/blob/master/sdks/python/apache_beam/runners/portability/portable_runner.py#L307 the call to Prepare() doesn't provide any timeout value and the same applies to other JobServer requests.
      As part of this ticket we could add a default timeout of 60 seconds as the default timeout for http client.
      Additionally, we could consider adding a --job-server-request-timeout to the PortableOptions class to be used in the JobServer interactions inside probable_runner.py.

      Attachments

        Issue Links

          Activity

            People

              enricoc Enrico Canzonieri
              enricoc Enrico Canzonieri
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 3h 50m
                  3h 50m