Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-2537

Can't run IO ITs - conflicting project vs projectId pipeline options

Details

    • Bug
    • Status: Resolved
    • P2
    • Resolution: Fixed
    • None
    • 2.1.0
    • io-java-gcp
    • None

    Description

      Today, the GPC IO ITs use a mix of --project and --projectId pipeline options.

      eg. GcpOptions.java uses --project:
      https://github.com/apache/beam/blob/master/sdks/java/extensions/google-cloud-platform-core/src/main/java/org/apache/beam/sdk/extensions/gcp/options/GcpOptions.java#L81

      and e.g the datastore IO ITs use that parameter:
      https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/datastore/V1ReadIT.java#L60

      however, BigtableTestOptions uses --projectId:
      https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/bigtable/BigtableTestOptions.java#L30

      as does the spanner IO IT:
      https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/test/java/org/apache/beam/sdk/io/gcp/spanner/SpannerReadIT.java#L61

      These conflicts mean that you can't run the tests as a suite successfully if you need to manually specify the project.

      This works with the integration tests currently checked in b/c they are using the default project values (apache-beam-testing) when running in CI. However, that's not an option for running these tests as part of a suite when doing local development.

      cc jasonkuster

      Attachments

        Issue Links

          Activity

            People

              sisk Stephen Sisk
              sisk Stephen Sisk
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: