Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-2523

GCP IO exposes protobuf on its API surface, causing user pain

Details

    • Bug
    • Status: Open
    • P3
    • Resolution: Unresolved
    • None
    • None
    • None

    Description

      Putting the SDK, DataflowRunner, and GCP IO on the same classpath, results in (at least) three versions of protobuf getting pulled in. These should be made to converge. We should consider using maven enforcer, which I think can check this.

      [INFO] com.example:foo:jar:0.1
      [INFO] +- org.apache.beam:beam-sdks-java-core:jar:2.0.0:compile
      [INFO] +- org.apache.beam:beam-sdks-java-io-google-cloud-platform:jar:2.0.0:compile
      [INFO] |  +- org.apache.beam:beam-sdks-java-extensions-protobuf:jar:2.0.0:compile
      [INFO] |  |  \- (com.google.protobuf:protobuf-java:jar:3.2.0:compile - omitted for duplicate)
      [INFO] |  +- com.google.api.grpc:grpc-google-pubsub-v1:jar:0.1.0:compile
      [INFO] |  |  +- (com.google.protobuf:protobuf-java:jar:3.0.0:compile - omitted for conflict with 3.2.0)
      [INFO] |  |  \- com.google.api.grpc:grpc-google-iam-v1:jar:0.1.0:compile
      [INFO] |  |     \- (com.google.protobuf:protobuf-java:jar:3.0.0:compile - omitted for conflict with 3.2.0)
      [INFO] |  +- com.google.cloud.datastore:datastore-v1-proto-client:jar:1.4.0:compile
      [INFO] |  |  +- (com.google.cloud.datastore:datastore-v1-protos:jar:1.3.0:compile - omitted for duplicate)
      [INFO] |  |  +- (com.google.http-client:google-http-client:jar:1.20.0:compile - omitted for conflict with 1.22.0)
      [INFO] |  |  +- com.google.http-client:google-http-client-protobuf:jar:1.20.0:compile
      [INFO] |  |  |  +- (com.google.http-client:google-http-client:jar:1.20.0:compile - omitted for conflict with 1.22.0)
      [INFO] |  |  |  \- (com.google.protobuf:protobuf-java:jar:2.4.1:compile - omitted for conflict with 3.2.0)
      [INFO] |  +- com.google.cloud.datastore:datastore-v1-protos:jar:1.3.0:compile
      [INFO] |  |  +- (com.google.protobuf:protobuf-java:jar:3.0.0:compile - omitted for conflict with 3.2.0)
      [INFO] |  +- com.google.cloud.bigtable:bigtable-protos:jar:0.9.6.2:compile
      [INFO] |  |  +- (com.google.code.findbugs:jsr305:jar:3.0.1:compile - omitted for duplicate)
      [INFO] |  |  +- (com.google.protobuf:protobuf-java:jar:3.2.0:compile - omitted for duplicate)
      

      Incidentally, the dependency plugin stopped supporting the verbose tree, so we can't even visually inspect this except by downgrading.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              kenn Kenneth Knowles
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated: