Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-14295

[Go SDK] Add flag for Dataflow Service Options to dataflow runner

Details

    • Wish
    • Status: Resolved
    • P2
    • Resolution: Fixed
    • None
    • 2.40.0
    • sdk-go

    Description

      This would allow the Go SDK to use dataflow service options for environment configuration

      eg. for enabling secure container boot:
      https://cloud.google.com/dataflow/docs/guides/deploying-a-pipeline#python_12

      They eventually get plumbed to here: https://pkg.go.dev/google.golang.org/api/dataflow/v1b3#Environment

      Ultimately, needs to be plumbed to the Environment.ServiceOptions field somewhere here:
      https://github.com/apache/beam/blob/716e3b1ddce527d6a10d054e55da55d80ebe8513/sdks/go/pkg/beam/runners/dataflow/dataflowlib/job.go#L157

      Simple enough: Add a flag that accepts a comma separated list of options, parse those to a []string, and populate

      Flag would go here: https://github.com/apache/beam/blob/716e3b1ddce527d6a10d054e55da55d80ebe8513/sdks/go/pkg/beam/runners/dataflow/dataflow.go#L53

      Attachments

        Activity

          People

            damccorm Danny McCormick
            lostluck Robert Burke
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: