Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-5605 Support Portable SplittableDoFn for batch
  3. BEAM-4737

SplittableDoFn dynamic rebalancing in Dataflow

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • P2
    • Resolution: Fixed
    • None
    • 2.21.0
    • runner-dataflow
    • None

    Description

      Related: https://issues.apache.org/jira/browse/BEAM-4287 and https://issues.apache.org/jira/browse/BEAM-4288 (language APIs for SDF splitting)

       

      The current Jira is to track supporting those APIs in Cloud Dataflow runner. Other runners don't have dynamic rebalancing (liquid sharding) so it doesn't apply to them.

      Attachments

        Issue Links

          is depended upon by

          Improvement - An improvement or enhancement to an existing feature or task. BEAM-4735 Make HBaseIO.read() based on SDF by default

          • P3 - Non-urgent bugs, features, and improvements
          • Open

          Activity

            People

              lcwik Luke Cwik
              jkff Eugene Kirpichov
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: