Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-10286

ExecutorServiceParallelExecutor.waitUntilFinish drops exceptions

Details

    • Bug
    • Status: Resolved
    • P1
    • Resolution: Fixed
    • None
    • 2.23.0
    • runner-direct
    • None

    Description

      This is a race, I have been unable to reproduce in an isolated test, but it is trivial to reproduce in a google internal integration suite with the following change to runners/direct-java/src/main/java/org/apache/beam/runners/direct/ExecutorServiceParallelExecutor.java

             // Get an update; don't block forever if another thread has handled it. The call to poll will
             // wait the entire timeout; this call primarily exists to relinquish any core.
             VisibleExecutorUpdate update = visibleUpdates.tryNext(Duration.millis(25L));
      +      Thread.sleep(100);
       
             if (update == null && pipelineState.get().isTerminal()) {
      

      Attachments

        Issue Links

          Activity

            People

              apilloud Andrew Pilloud
              apilloud Andrew Pilloud
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 50m
                  50m