Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-7688

Flink portable runner gets stuck when waiting for SDK Harness to close

Details

    • Bug
    • Status: Resolved
    • P2
    • Resolution: Duplicate
    • None
    • Not applicable
    • runner-flink
    • None

    Description

      When parallelism = nproc:

      "MapPartition (MapPartition at [37]

      {Analyze, RandomizeData, ReadFromText, DecodeForAnalyze}

      ) (9/12)" #2855 prio=5 os_prio=0 tid=0x00007f9184022800 nid=0x2b58 waiting on condition [0x00007f9091592000]
      java.lang.Thread.State: WAITING (parking)
      at (C/C++) 0x00007f926a97a9f2 (Unknown Source)
      at (C/C++) 0x00007f9269f1dd99 (Unknown Source)
      at sun.misc.Unsafe.park(Native Method)

      • parking to wait for <0x00000000ca218030> (a java.util.concurrent.CompletableFuture$Signaller)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1693)
        at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
        at java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1729)
        at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
        at org.apache.beam.sdk.fn.data.CompletableFutureInboundDataClient.awaitCompletion(CompletableFutureInboundDataClient.java:48)
        at org.apache.beam.sdk.fn.data.BeamFnDataInboundObserver.awaitCompletion(BeamFnDataInboundObserver.java:90)
        at org.apache.beam.runners.fnexecution.control.SdkHarnessClient$ActiveBundle.close(SdkHarnessClient.java:298)
        at org.apache.beam.runners.flink.translation.functions.FlinkExecutableStageFunction.$closeResource(FlinkExecutableStageFunction.java:209)
        at org.apache.beam.runners.flink.translation.functions.FlinkExecutableStageFunction.mapPartition(FlinkExecutableStageFunction.java:209)
        at org.apache.flink.runtime.operators.MapPartitionDriver.run(MapPartitionDriver.java:103)
        at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503)
        at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368)
        at org.apache.flink.runtime.taskmanager.Task.run(Task.java:712)
        at java.lang.Thread.run(Thread.java:748)

      Attachments

        Issue Links

          Activity

            People

              ibzib Kyle Weaver
              ibzib Kyle Weaver
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: