Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-8668

TypeError: _create_function() takes from 2 to 6 positional arguments but 7 were given

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: sdk-py-harness
    • Labels:
      None

      Description

      This error looks to me a lot like a mismatch between SDK versions (Dill dependencies), but I've had a couple reports of it so I thought it worth making it discoverable here.

       
      Traceback (most recent call last):
      File "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py", line 158, in _execute
      response = task()
      File "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py", line 191, in <lambda>
      self._execute(lambda: worker.do_instruction(work), work)
      File "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py", line 343, in do_instruction
      request.instruction_id)
      File "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py", line 363, in process_bundle
      instruction_id, request.process_bundle_descriptor_reference)
      File "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/sdk_worker.py", line 306, in get
      self.data_channel_factory)
      File "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py", line 578, in _init_
      self.ops = self.create_execution_tree(self.process_bundle_descriptor)
      File "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py", line 622, in create_execution_tree
      descriptor.transforms, key=topological_height, reverse=True)])
      File "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py", line 621, in <listcomp>
      for transform_id in sorted(
      File "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py", line 546, in wrapper
      result = cache[args] = func(*args)
      File "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py", line 605, in get_operation
      in descriptor.transforms[transform_id].outputs.items()
      File "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py", line 604, in <dictcomp>
      for tag, pcoll_id
      File "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py", line 603, in <listcomp>
      tag: [get_operation(op) for op in pcoll_consumers[pcoll_id]]
      File "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py", line 546, in wrapper
      result = cache[args] = func(*args)
      File "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py", line 608, in get_operation
      transform_id, transform_consumers)
      File "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py", line 867, in create_operation
      return creator(self, transform_id, transform_proto, payload, consumers)
      File "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1110, in create
      serialized_fn, parameter)
      File "/usr/local/lib/python3.6/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1148, in _create_pardo_operation
      dofn_data = pickler.loads(serialized_fn)
      File "/usr/local/lib/python3.6/site-packages/apache_beam/internal/pickler.py", line 265, in loads
      return dill.loads(s)
      File "/usr/local/lib/python3.6/site-packages/dill/_dill.py", line 317, in loads
      return load(file, ignore)
      File "/usr/local/lib/python3.6/site-packages/dill/_dill.py", line 305, in load
      obj = pik.load()
      TypeError: _create_function() takes from 2 to 6 positional arguments but 7 were given
       

        Attachments

          Activity

            People

            • Assignee:
              ibzib Kyle Weaver
              Reporter:
              ibzib Kyle Weaver
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: