Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-7472

Update SplittableDoFn implementation within Python to use ProcessContinuation instead of defer_remainder/defer_status on the RestrictionTracker

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: sdk-py-core, sdk-py-harness
    • Labels:
      None

      Description

      The current API for a SplittableDoFn doesn't follow the API in https://docs.google.com/document/d/1h_zprJrOilivK2xfvl4L42vaX4DMYGfH1YDmi-s_ozM/edit?usp=sharing

      Specifically the intent was that a ProcessContinuation was to be returned as the last result for a a SplittableDoFn signalling the users intent as to whether the restriction is done or whether it should be resumed after a certain amount of time.

       

      This requires updating the Python SDK implementation to not rely on RestrictionTracker.defer_remainder and RestrictionTracker.defer_status

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              lcwik Luke Cwik
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: