Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-7989

SparkRunner CacheVisitor counts PCollections from SideInputs

Details

    • Bug
    • Status: Resolved
    • P2
    • Resolution: Fixed
    • 2.14.0
    • 2.16.0
    • runner-spark
    • None

    Description

      The SparkRunner's CacheVisitor looks at all inputs for a TransformHierarchy.Node. Those inputs include the PCollections from the PCollectionViews that are supplied as sideInputs.

      The SparkRunner should not count these instances of sideInputs as the PCollections are not actually accessed. They are only accessed when the CreatePCollectionView Transform is processed.

      Attachments

        Issue Links

          Activity

            People

              winkelman.kyle Kyle Winkelman
              winkelman.kyle Kyle Winkelman
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 10m
                  1h 10m