Details
-
Bug
-
Status: Resolved
-
P2
-
Resolution: Cannot Reproduce
-
None
Description
Use this form to file an issue for test failure:
Initial investigation:
12:08:13 ERROR: test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT)
12:08:13 ----------------------------------------------------------------------
12:08:13 Traceback (most recent call last):
12:08:13 File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_ValCont/src/sdks/python/apache_beam/examples/wordcount_it_test.py", line 52, in test_wordcount_fnapi_it
12:08:13 self._run_wordcount_it(wordcount.run, experiment='beam_fn_api')
12:08:13 File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_ValCont/src/sdks/python/apache_beam/examples/wordcount_it_test.py", line 84, in _run_wordcount_it
12:08:13 run_wordcount(test_pipeline.get_full_options_as_args(**extra_opts))
12:08:13 File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_ValCont/src/sdks/python/apache_beam/examples/wordcount.py", line 114, in run
12:08:13 result = p.run()
12:08:13 File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_ValCont/src/sdks/python/apache_beam/pipeline.py", line 406, in run
12:08:13 self._options).run(False)
12:08:13 File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_ValCont/src/sdks/python/apache_beam/pipeline.py", line 419, in run
12:08:13 return self.runner.run_pipeline(self, self._options)
12:08:13 File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_ValCont/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py", line 64, in run_pipeline
12:08:13 self.result.wait_until_finish(duration=wait_duration)
12:08:13 File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_ValCont/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py", line 1338, in wait_until_finish
12:08:13 (self.state, getattr(self._runner, 'last_error_msg', None)), self)
12:08:13 DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
12:08:13 java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error received from SDK harness for instruction -129: Traceback (most recent call last):
12:08:13 File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 157, in _execute
12:08:13 response = task()
12:08:13 File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 190, in <lambda>
12:08:13 self._execute(lambda: worker.do_instruction(work), work)
12:08:13 File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 342, in do_instruction
12:08:13 request.instruction_id)
12:08:13 File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 368, in process_bundle
12:08:13 bundle_processor.process_bundle(instruction_id))
12:08:13 File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 593, in process_bundle
12:08:13 data.ptransform_id].process_encoded(data.data)
12:08:13 KeyError: u'\n\x04-107\x12\x04-105'
12:08:13
12:08:13 at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
12:08:13 at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
12:08:13 at org.apache.beam.sdk.util.MoreFutures.get(MoreFutures.java:57)
12:08:13 at org.apache.beam.runners.dataflow.worker.fn.control.RegisterAndProcessBundleOperation.finish(RegisterAndProcessBundleOperation.java:285)
12:08:13 at org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:85)
12:08:13 at org.apache.beam.runners.dataflow.worker.fn.control.BeamFnMapTaskExecutor.execute(BeamFnMapTaskExecutor.java:125)
12:08:13 at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:412)
12:08:13 at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:381)
12:08:13 at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:306)
12:08:13 at org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.start(DataflowRunnerHarness.java:195)
12:08:13 at org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.main(DataflowRunnerHarness.java:123)
12:08:13 Suppressed: java.lang.IllegalStateException: Already closed.
12:08:13 at org.apache.beam.sdk.fn.data.BeamFnDataBufferingOutboundObserver.close(BeamFnDataBufferingOutboundObserver.java:95)
12:08:13 at org.apache.beam.runners.dataflow.worker.fn.data.RemoteGrpcPortWriteOperation.abort(RemoteGrpcPortWriteOperation.java:215)
12:08:13 at org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:91)
12:08:13 ... 6 more
12:08:13 Caused by: java.lang.RuntimeException: Error received from SDK harness for instruction -129: Traceback (most recent call last):
12:08:13 File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 157, in _execute
12:08:13 response = task()
12:08:13 File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 190, in <lambda>
12:08:13 self._execute(lambda: worker.do_instruction(work), work)
12:08:13 File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 342, in do_instruction
12:08:13 request.instruction_id)
12:08:13 File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 368, in process_bundle
12:08:13 bundle_processor.process_bundle(instruction_id))
12:08:13 File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 593, in process_bundle
12:08:13 data.ptransform_id].process_encoded(data.data)
12:08:13 KeyError: u'\n\x04-107\x12\x04-105'
12:08:13
12:08:13 at org.apache.beam.runners.fnexecution.control.FnApiControlClient$ResponseStreamObserver.onNext(FnApiControlClient.java:157)
12:08:13 at org.apache.beam.runners.fnexecution.control.FnApiControlClient$ResponseStreamObserver.onNext(FnApiControlClient.java:140)
12:08:13 at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.stub.ServerCalls$StreamingServerCallHandler$StreamingServerCallListener.onMessage(ServerCalls.java:248)
12:08:13 at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.ForwardingServerCallListener.onMessage(ForwardingServerCallListener.java:33)
12:08:13 at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.Contexts$ContextualizedServerCallListener.onMessage(Contexts.java:76)
12:08:13 at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.messagesAvailable(ServerCallImpl.java:263)
12:08:13 at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1MessagesAvailable.runInContext(ServerImpl.java:683)
12:08:13 at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
12:08:13 at org.apache.beam.vendor.grpc.v1p13p1.io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
12:08:13 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
12:08:13 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
12:08:13 at java.lang.Thread.run(Thread.java:745)
After you've filled out the above details, please assign the issue to an individual. Assignee should treat test failures as high-priority, helping to fix the issue or find a more appropriate owner. See Apache Beam Post-Commit Policies.
Attachments
Issue Links
- is blocked by
-
BEAM-7063 Stage Dataflow runner harness in Dataflow FnApi Python test suites that run using an unreleased SDK.
- Resolved