Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-7797

DataflowRunner's upload_graph feature doesn't reduce template file size

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.15.0
    • Component/s: runner-dataflow
    • Labels:
      None

      Description

      Dataflow Runner has a feature to save job graph separately, reducing the size of the request to Dataflow API.

      Since templates are simply dumps of requests to the API, sizes of template files should be reduced as well.

       

      However, the logic to remove parts from the request happened after template file generation.

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                azurezyq Yunqing Zhou
                Reporter:
                azurezyq Yunqing Zhou
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - 48h
                  48h
                  Remaining:
                  Time Spent - 50m Remaining Estimate - 47h 10m
                  47h 10m
                  Logged:
                  Time Spent - 50m Remaining Estimate - 47h 10m
                  50m