Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-9081

Fail Dataflow Streaming pipeline on passert failure

Details

    • Improvement
    • Status: Open
    • P3
    • Resolution: Unresolved
    • None
    • None
    • sdk-py-core, testing
    • None

    Description

      Dataflow retries a failed work item indefinitely.

      As PAsserts are translated to PTransforms, A failed PAssert will be retried indefinitely in Streaming mode.

      So we should stop fail the pipeline in case of PAssert failure instead of waiting for test timeout.

       

      Example job https://pantheon.corp.google.com/dataflow/jobsDetail/locations/us-west1/jobs/2020-01-09_13_25_08-2233620052523810338?project=apache-beam-testing&folder&organizationId

      Attachments

        Activity

          People

            Unassigned Unassigned
            angoenka Ankur Goenka
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: