Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-14128

Quadratic behavior in go pipeline rewriting.

Details

    • Bug
    • Status: Triage Needed
    • P2
    • Resolution: Fixed
    • 2.37.0
    • 2.38.0
    • sdk-go
    • None

    Description

      Work is repeated for every node outside of a given composite, when checking if anything is using any outputs of transforms internal to the composite.

      This bug significantly increases the construction time large pipeline graphs, but can be resolved with bit of pre-computation and plumbing of which transforms are using which PCollections.

      Attachments

        Issue Links

          Activity

            People

              lostluck Robert Burke
              lostluck Robert Burke
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 10m
                  1h 10m