Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-12351

combine should be parallelizable in many cases

Details

    Description

      Relevant discussion: https://lists.apache.org/thread.html/r9e7d9527eb1d4c9c097c91c010a25dabf4a5f8053d50dc3b6d90d36a%40%3Cdev.beam.apache.org%3E

      Currently we require Singleton partitioning for combine() because func might operate on the full dataset, but in many cases func is actually an elementwise method. We should detect this when possible (e.g. when func is an np.ufunc), and/or provide a flag to let the user indicate the function is elementwise.

      Attachments

        Activity

          People

            Unassigned Unassigned
            bhulette Brian Hulette
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: