Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-14410

FnRunnerTest with non-trivial (order 1000 elements) numpy input flakes in non-cython environment

Details

    • Bug
    • Status: Open
    • P2
    • Resolution: Unresolved
    • None
    • None
    • sdk-py-core

    Description

      See https://github.com/apache/beam/pull/17384#issuecomment-1117418018

       

       apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:141: 
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      apache_beam/pipeline.py:596: in __exit__
          self.result = self.run()
      apache_beam/pipeline.py:573: in run
          return self.runner.run_pipeline(self, self._options)
      apache_beam/runners/portability/fn_api_runner/fn_runner.py:200: in run_pipeline
          pipeline.to_runner_api(default_environment=self._default_environment))
      apache_beam/runners/portability/fn_api_runner/fn_runner.py:208: in run_via_runner_api
          return self.run_stages(stage_context, stages)
      apache_beam/runners/portability/fn_api_runner/fn_runner.py:409: in run_stages
          runner_execution_context, bundle_context_manager, bundle_input)
      apache_beam/runners/portability/fn_api_runner/fn_runner.py:742: in _execute_bundle
          bundle_manager))
      apache_beam/runners/portability/fn_api_runner/fn_runner.py:966: in _run_bundle
          data_input, data_output, input_timers, expected_timer_output)
      apache_beam/runners/portability/fn_api_runner/fn_runner.py:1309: in process_bundle
          output.transform_id).append(output.data)
      apache_beam/runners/portability/fn_api_runner/execution.py:242: in append
          windowed_key_value = coder_impl.decode_from_stream(input_stream, True)
      apache_beam/coders/coder_impl.py:1446: in decode_from_stream
          value = self._value_coder.decode_from_stream(in_stream, nested)
      apache_beam/coders/coder_impl.py:994: in decode_from_stream
          c in enumerate(self._coder_impls)
      apache_beam/coders/coder_impl.py:994: in <listcomp>
          c in enumerate(self._coder_impls)
      apache_beam/coders/coder_impl.py:617: in decode_from_stream
          return in_stream.read_all(nested)
      apache_beam/coders/slow_stream.py:137: in read_all
          return self.read(self.read_var_int64() if nested else self.size())
      apache_beam/coders/slow_stream.py:148: in read_var_int64
          byte = self.read_byte()
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <apache_beam.coders.slow_stream.InputStream object at 0x7f8433d63190>    def read_byte(self):
            # type: () -> int
            self.pos += 1
      >     return self.data[self.pos - 1]
      E     IndexError: index out of rangeapache_beam/coders/slow_stream.py:142: IndexError

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              bhulette Brian Hulette
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 20m
                  1h 20m