Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-13385

Test non default pickle library

Details

    • Improvement
    • Status: Open
    • P3
    • Resolution: Unresolved
    • None
    • None
    • sdk-py-core
    • None

    Description

      The python SDK recently added support for the cloudpickle library though the default is still dill. See [_https://s.apache.org/beam-picklers_] and https://issues.apache.org/jira/browse/BEAM-8123.

       

      We intend to switch this over to cloudpickle as the default.

       

      We should have a suite of E2E tests to make sure we don't introduce regressions into whichever pickle library is not the default.  That is to say:

      First -  E2E tests for the using the cloudpickle library.

      After cloudpickle library becomes the default - switch these E2E tests to cover dill.

       

      This will allow test coverage while we hopefully transition to just supporting a single library.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              Ryan.Thompson Ryan Thompson
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: