Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-7933

Adding timeout to JobServer grpc calls

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: In Progress
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.14.0
    • Fix Version/s: None
    • Component/s: sdk-py-core
    • Labels:

      Description

      grpc calls to the JobServer from the Python SDK do not have timeouts. That means that the call to pipeline.run()could hang forever if the JobServer is not running (or failing to start).
      E.g. https://github.com/apache/beam/blob/master/sdks/python/apache_beam/runners/portability/portable_runner.py#L307 the call to Prepare() doesn't provide any timeout value and the same applies to other JobServer requests.
      As part of this ticket we could add a default timeout of 60 seconds as the default timeout for http client.
      Additionally, we could consider adding a --job-server-request-timeout to the PortableOptions class to be used in the JobServer interactions inside probable_runner.py.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                enricoc Enrico Canzonieri
                Reporter:
                enricoc Enrico Canzonieri
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 3h 50m
                  3h 50m