Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-8193 Python PostCommit Tests are Flaky
  3. BEAM-8326

Run Portable_Python PreCommit fails with pickling

Details

    • Sub-task
    • Status: Resolved
    • P2
    • Resolution: Duplicate
    • None
    • Not applicable
    • sdk-py-core
    • None

    Description

      Test log: https://builds.apache.org/job/beam_PreCommit_Portable_Python_Phrase/434/console

      I see the following error:

      09:31:34 Traceback (most recent call last):
      09:31:34 File "/usr/local/lib/python3.5/site-packages/apache_beam/internal/pickler.py", line 261, in loads
      09:31:34 return dill.loads(s)
      09:31:34 File "/usr/local/lib/python3.5/site-packages/dill/_dill.py", line 317, in loads
      09:31:34 return load(file, ignore)
      09:31:34 File "/usr/local/lib/python3.5/site-packages/dill/_dill.py", line 305, in load
      09:31:34 obj = pik.load()
      09:31:34 TypeError: _create_function() takes from 2 to 6 positional arguments but 7 were given
      09:31:34
      09:31:34 During handling of the above exception, another exception occurred:
      09:31:34
      09:31:34 Traceback (most recent call last):
      09:31:34 File "/usr/local/lib/python3.5/site-packages/apache_beam/runners/worker/sdk_worker.py", line 158, in _execute
      09:31:34 response = task()
      09:31:34 File "/usr/local/lib/python3.5/site-packages/apache_beam/runners/worker/sdk_worker.py", line 191, in <lambda>
      09:31:34 self._execute(lambda: worker.do_instruction(work), work)
      09:31:34 File "/usr/local/lib/python3.5/site-packages/apache_beam/runners/worker/sdk_worker.py", line 343, in do_instruction
      09:31:34 request.instruction_id)
      09:31:34 File "/usr/local/lib/python3.5/site-packages/apache_beam/runners/worker/sdk_worker.py", line 363, in process_bundle
      09:31:34 instruction_id, request.process_bundle_descriptor_id)
      09:31:34 File "/usr/local/lib/python3.5/site-packages/apache_beam/runners/worker/sdk_worker.py", line 306, in get
      09:31:34 self.data_channel_factory)
      09:31:34 File "/usr/local/lib/python3.5/site-packages/apache_beam/runners/worker/bundle_processor.py", line 580, in _init_
      09:31:34 self.ops = self.create_execution_tree(self.process_bundle_descriptor)
      09:31:34 File "/usr/local/lib/python3.5/site-packages/apache_beam/runners/worker/bundle_processor.py", line 624, in create_execution_tree
      09:31:34 descriptor.transforms, key=topological_height, reverse=True)])
      09:31:34 File "/usr/local/lib/python3.5/site-packages/apache_beam/runners/worker/bundle_processor.py", line 623, in <listcomp>
      09:31:34 for transform_id in sorted(
      09:31:34 File "/usr/local/lib/python3.5/site-packages/apache_beam/runners/worker/bundle_processor.py", line 548, in wrapper
      09:31:34 result = cache[args] = func(*args)
      09:31:34 File "/usr/local/lib/python3.5/site-packages/apache_beam/runners/worker/bundle_processor.py", line 607, in get_operation
      09:31:34 in descriptor.transforms[transform_id].outputs.items()
      09:31:34 File "/usr/local/lib/python3.5/site-packages/apache_beam/runners/worker/bundle_processor.py", line 606, in <dictcomp>
      09:31:34 for tag, pcoll_id
      09:31:34 File "/usr/local/lib/python3.5/site-packages/apache_beam/runners/worker/bundle_processor.py", line 605, in <listcomp>
      09:31:34 tag: [get_operation(op) for op in pcoll_consumers[pcoll_id]]
      09:31:34 File "/usr/local/lib/python3.5/site-packages/apache_beam/runners/worker/bundle_processor.py", line 548, in wrapper
      09:31:34 result = cache[args] = func(*args)
      09:31:34 File "/usr/local/lib/python3.5/site-packages/apache_beam/runners/worker/bundle_processor.py", line 610, in get_operation
      09:31:34 transform_id, transform_consumers)
      09:31:34 File "/usr/local/lib/python3.5/site-packages/apache_beam/runners/worker/bundle_processor.py", line 869, in create_operation
      09:31:34 return creator(self, transform_id, transform_proto, payload, consumers)
      09:31:34 File "/usr/local/lib/python3.5/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1112, in create
      09:31:34 serialized_fn, parameter)
      09:31:34 File "/usr/local/lib/python3.5/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1150, in _create_pardo_operation
      09:31:34 dofn_data = pickler.loads(serialized_fn)
      09:31:34 File "/usr/local/lib/python3.5/site-packages/apache_beam/internal/pickler.py", line 265, in loads
      09:31:34 return dill.loads(s)
      09:31:34 File "/usr/local/lib/python3.5/site-packages/dill/_dill.py", line 317, in loads
      09:31:34 return load(file, ignore)
      09:31:34 File "/usr/local/lib/python3.5/site-packages/dill/_dill.py", line 305, in load
      09:31:34 obj = pik.load()
      09:31:34 TypeError: _create_function() takes from 2 to 6 positional arguments but 7 were given

      First exception `IndexError` is expected and handled here (https://github.com/apache/beam/blob/ee45cb03f6e6f3685ae0dccf75a8af4d259b3b1a/sdks/python/apache_beam/runners/worker/sdk_worker.py#L301)

      The issue here is with pickling, might be a dill issue.

      Attachments

        Issue Links

          Activity

            People

              tvalentyn Valentyn Tymofieiev
              altay Ahmet Altay
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: