beam_PostCommit_Python36 - CrossLanguageSpannerIOTest - flakey failing

Use this form to file an issue for test failure

_https://ci-beam.apache.org/job/beam_PostCommit_Python36/4385/_

• apache_beam.io.gcp.tests.xlang_spannerio_it_test.CrossLanguageSpannerIOTest.test_spanner_read_query
• apache_beam.io.gcp.tests.xlang_spannerio_it_test.CrossLanguageSpannerIOTest.test_spanner_read_table

Initial investigation:

Error Message
RuntimeError: Pipeline BeamApp-jenkins-0921123257-8b921bc2_f20c9b92-9261-4f0b-9d6c-31732a12287b failed in state FAILED: akka.pattern.AskTimeoutException: Ask timed out on Actor[akka://flink/user/rpc/taskmanager_0#31887928] after [10000 ms]. Message of type [org.apache.flink.runtime.rpc.messages.LocalRpcInvocation]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply.
Stacktrace
self = <apache_beam.io.gcp.tests.xlang_spannerio_it_test.CrossLanguageSpannerIOTest testMethod=test_spanner_read_query>

def test_spanner_read_query(self):
self.insert_read_values('query_read')
> self.run_read_pipeline('query_read', query=f'SELECT * FROM {self.table}')

apache_beam/io/gcp/tests/xlang_spannerio_it_test.py:194:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/io/gcp/tests/xlang_spannerio_it_test.py:220: in run_read_pipeline
SpannerTestRow(f_int64=2, f_string=f'{prefix}2', f_boolean=False),
apache_beam/pipeline.py:590: in _exit_
self.result = self.run()
apache_beam/testing/test_pipeline.py:116: in run
state = result.wait_until_finish()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <apache_beam.runners.portability.portable_runner.PipelineResult object at 0x7fa74c495860>
duration = None

def wait_until_finish(self, duration=None):
"""
:param duration: The maximum time in milliseconds to wait for the result of
the execution. If None or zero, will wait until the pipeline finishes.
:return: The result of the pipeline, i.e. PipelineResult.
"""
def read_messages():

1. type: () -> None
   previous_state = -1
   for message in self._message_stream:
   if message.HasField('message_response'):
   logging.log(
   MESSAGE_LOG_LEVELS[message.message_response.importance],
   "%s",
   message.message_response.message_text)
   else:
   current_state = message.state_response.state
   if current_state != previous_state:
   _LOGGER.info(
   "Job state changed to %s",
   self._runner_api_state_to_pipeline_state(current_state))
   previous_state = current_state
   self._messages.append(message)

message_thread = threading.Thread(
target=read_messages, name='wait_until_finish_read')
message_thread.daemon = True
message_thread.start()

if duration:
state_thread = threading.Thread(
target=functools.partial(self._observe_state, message_thread),
name='wait_until_finish_state_observer')
state_thread.daemon = True
state_thread.start()
start_time = time.time()
duration_secs = duration / 1000
while (time.time() - start_time < duration_secs and
state_thread.is_alive()):
time.sleep(1)
else:
self._observe_state(message_thread)

if self._runtime_exception:
> raise self._runtime_exception
E RuntimeError: Pipeline BeamApp-jenkins-0921123257-8b921bc2_f20c9b92-9261-4f0b-9d6c-31732a12287b failed in state FAILED: akka.pattern.AskTimeoutException: Ask timed out on Actor[akka://flink/user/rpc/taskmanager_0#31887928] after [10000 ms]. Message of type [org.apache.flink.runtime.rpc.messages.LocalRpcInvocation]. A typical reason for `AskTimeoutException` is that the recipient actor didn't send a reply.

apache_beam/runners/portability/portable_runner.py:600: RuntimeError

After you've filled out the above details, please assign the issue to an individual. Assignee should treat test failures as high-priority, helping to fix the issue or find a more appropriate owner. See Apache Beam Post-Commit Policies.