Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-7510

test_write_to_different_file_types is flaky

Details

    • Bug
    • Status: Resolved
    • P2
    • Resolution: Fixed
    • None
    • 2.14.0
    • io-py-gcp
    • None

    Description

      10:54:51 ======================================================================
      10:54:51 ERROR: test_write_to_different_file_types (apache_beam.io.fileio_test.WriteFilesTest)
      10:54:51 ----------------------------------------------------------------------
      10:54:51 Traceback (most recent call last):
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/io/fileio_test.py", line 420, in test_write_to_different_file_types
      10:54:51     label='verifyApache')
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/pipeline.py", line 426, in __exit__
      10:54:51     self.run().wait_until_finish()
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/testing/test_pipeline.py", line 107, in run
      10:54:51     else test_runner_api))
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/pipeline.py", line 406, in run
      10:54:51     self._options).run(False)
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/pipeline.py", line 419, in run
      10:54:51     return self.runner.run_pipeline(self, self._options)
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/direct/direct_runner.py", line 128, in run_pipeline
      10:54:51     return runner.run_pipeline(pipeline, options)
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner.py", line 289, in run_pipeline
      10:54:51     default_environment=self._default_environment))
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner.py", line 293, in run_via_runner_api
      10:54:51     return self.run_stages(*self.create_stages(pipeline_proto))
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner.py", line 369, in run_stages
      10:54:51     stage_context.safe_coders)
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner.py", line 533, in run_stage
      10:54:51     data_input, data_output)
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner.py", line 1239, in process_bundle
      10:54:51     result_future = self._controller.control_handler.push(process_bundle)
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/portability/fn_api_runner.py", line 854, in push
      10:54:51     response = self.worker.do_instruction(request)
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/worker/sdk_worker.py", line 342, in do_instruction
      10:54:51     request.instruction_id)
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/worker/sdk_worker.py", line 368, in process_bundle
      10:54:51     bundle_processor.process_bundle(instruction_id))
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/worker/bundle_processor.py", line 589, in process_bundle
      10:54:51     ].process_encoded(data.data)
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/worker/bundle_processor.py", line 143, in process_encoded
      10:54:51     self.output(decoded_value)
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/worker/operations.py", line 256, in output
      10:54:51     cython.cast(Receiver, self.receivers[output_index]).receive(windowed_value)
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/worker/operations.py", line 143, in receive
      10:54:51     self.consumer.process(windowed_value)
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/worker/operations.py", line 594, in process
      10:54:51     delayed_application = self.dofn_receiver.receive(o)
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 778, in receive
      10:54:51     self.process(windowed_value)
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 784, in process
      10:54:51     self._reraise_augmented(exn)
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 782, in process
      10:54:51     return self.do_fn_invoker.invoke_process(windowed_value)
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 595, in invoke_process
      10:54:51     windowed_value, additional_args, additional_kwargs, output_processor)
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 666, in _invoke_process_per_window
      10:54:51     windowed_value, self.process_method(*args_for_process))
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 921, in process_outputs
      10:54:51     self.main_receivers.receive(windowed_value)
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/worker/operations.py", line 143, in receive
      10:54:51     self.consumer.process(windowed_value)
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/worker/operations.py", line 594, in process
      10:54:51     delayed_application = self.dofn_receiver.receive(o)
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 778, in receive
      10:54:51     self.process(windowed_value)
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 784, in process
      10:54:51     self._reraise_augmented(exn)
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 782, in process
      10:54:51     return self.do_fn_invoker.invoke_process(windowed_value)
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 454, in invoke_process
      10:54:51     windowed_value, self.process_method(windowed_value.value))
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 921, in process_outputs
      10:54:51     self.main_receivers.receive(windowed_value)
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/worker/operations.py", line 143, in receive
      10:54:51     self.consumer.process(windowed_value)
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/worker/operations.py", line 594, in process
      10:54:51     delayed_application = self.dofn_receiver.receive(o)
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 778, in receive
      10:54:51     self.process(windowed_value)
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 784, in process
      10:54:51     self._reraise_augmented(exn)
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 851, in _reraise_augmented
      10:54:51     raise_with_traceback(new_exn)
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/target/.tox-py35-gcp/py35-gcp/lib/python3.5/site-packages/future/utils/__init__.py", line 419, in raise_with_traceback
      10:54:51     raise exc.with_traceback(traceback)
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 782, in process
      10:54:51     return self.do_fn_invoker.invoke_process(windowed_value)
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/runners/common.py", line 454, in invoke_process
      10:54:51     windowed_value, self.process_method(windowed_value.value))
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/transforms/core.py", line 1293, in <lambda>
      10:54:51     wrapper = lambda x: [fn(x)]
      10:54:51   File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/test-suites/tox/py35/build/srcs/sdks/python/apache_beam/testing/util.py", line 130, in _equal
      10:54:51     'Failed assert: %r == %r' % (sorted_expected, sorted_actual))
      10:54:51 apache_beam.testing.util.BeamAssertException: Failed assert: ['{"foundation": "cncf", "project": "grpc"}', '{"foundation": "cncf", "project": "knative"}', '{"foundation": "cncf", "project": "kubernetes"}', '{"foundation": "cncf", "project": "prometheus"}'] == ['{"project": "grpc", "foundation": "cncf"}', '{"project": "knative", "foundation": "cncf"}', '{"project": "kubernetes", "foundation": "cncf"}', '{"project": "prometheus", "foundation": "cncf"}'] [while running 'verifyCNCF/Match']
      

      Attachments

        Activity

          People

            pabloem Pablo Estrada
            tvalentyn Valentyn Tymofieiev
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 1h
                1h