Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-12794

PortableRunnerTestWithExternalEnv.test_pardo_timers flaky

Details

    • Bug
    • Status: Open
    • P1
    • Resolution: Unresolved
    • None
    • None

    Description

      In python precommit.

      Looks like there's also a bug in error handling. IIUC, this call is intended to be a "cast" but really it's calling an arbitrary constructor, which may take different arguments than a
      BaseException. https://github.com/apache/beam/blob/4559c75863d9d6c9dd9e48c2b4f12f2139410524/sdks/python/apache_beam/runners/worker/sdk_worker.py#L1105 cctvalentyn yoshiki.obata
       
      https://ci-beam.apache.org/job/beam_PreCommit_Python_Phrase/2755/testReport/junit/apache_beam.runners.portability.portable_runner_test/PortableRunnerTestWithExternalEnv/test_pardo_timers/
       
      E RuntimeError: Pipeline job-721c3984-7f9c-4991-8473-c7d8c02f4d13 failed in state FAILED: Error running pipeline. E Traceback (most recent call last): E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/portability/local_job_service.py", line 274, in run_job E self._pipeline_proto) E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner/fn_runner.py", line 206, in run_via_runner_api E return self.run_stages(stage_context, stages) E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner/fn_runner.py", line 385, in run_stages E runner_execution_context, bundle_context_manager) E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner/fn_runner.py", line 653, in _run_stage E bundle_manager)) E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner/fn_runner.py", line 770, in _run_bundle E data_input, data_output, input_timers, expected_timer_output) E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner/fn_runner.py", line 1120, in process_bundle E raise RuntimeError(result.error) E RuntimeError: Traceback (most recent call last): E File "apache_beam/runners/common.py", line 1258, in apache_beam.runners.common.DoFnRunner.process_user_timer E self.do_fn_invoker.invoke_user_timer( E File "apache_beam/runners/common.py", line 528, in apache_beam.runners.common.DoFnInvoker.invoke_user_timer E self.output_processor.process_outputs( E File "apache_beam/runners/common.py", line 1368, in apache_beam.runners.common._OutputProcessor.process_outputs E for result in results: E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner/fn_runner_test.py", line 366, in process_timer E if state.read() == 0: E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/worker/bundle_processor.py", line 481, in read E return self._combinefn.extract_output(self._read_accumulator()) E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/worker/bundle_processor.py", line 473, in _read_accumulator E self._underlying_bag_state.read()) E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/transforms/core.py", line 1038, in merge_accumulators E return [self._fn(_ReiterableChain(accumulators), *args, **kwargs)] E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/transforms/core.py", line 976, in __iter_ E for iterable in self.iterables: E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/worker/bundle_processor.py", line 521, in _iter_ E for elem in self.first: E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/worker/bundle_processor.py", line 353, in _iter_ E self.state_handler.blocking_get(self._state_key, self._coder_impl)) E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/worker/sdk_worker.py", line 1194, in blocking_get E self._partially_cached_iterable(state_key, coder)) E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/worker/sdk_worker.py", line 1300, in _partially_cached_iterable E data, continuation_token = self._underlying.get_raw(state_key, None) E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/worker/sdk_worker.py", line 1067, in get_raw E continuation_token=continuation_token))) E File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Phrase/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/worker/sdk_worker.py", line 1105, in _blocking_request E raise t(v).with_traceback(tb) E TypeError: __init_() missing 3 required positional arguments: 'call', 'response_deserializer', and 'deadline'

      Attachments

        Activity

          People

            Unassigned Unassigned
            ibzib Kyle Weaver
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 9h 10m
                9h 10m