Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-8497

Vendor or shade the dependencies of beam-sdks-java-io-google-cloud-platform

Details

    • Improvement
    • Status: Open
    • P3
    • Resolution: Unresolved
    • None
    • None
    • io-java-gcp
    • None

    Description

      Here at Spotify, we regularly face conflicting dependencies issues, usually caused by guava, grpc-* and netty-*. Looking at this bug tracker, it seems that we are not alone and that conflicting versions are a recurring issue.

      In beam-sdks-java-core, beam-runners-direct-java and beam-runners-google-cloud-dataflow-java those dependencies are vendored to avoid such issue.

      I suggest to do the same for beam-sdks-java-io-google-cloud-platform.

       

      FIY, here's the list of all the dependencies pulled by beam-sdks-java-io-google-cloud-platform:

      com.fasterxml.jackson.core:jackson-annotations:2.9.10:default
      com.fasterxml.jackson.core:jackson-core:2.9.10:default
      com.fasterxml.jackson.core:jackson-databind:2.9.10:default
      com.google.api:api-common:1.7.0:default
      com.google.api:gax:1.38.0:default
      com.google.api:gax-grpc:1.38.0:default
      com.google.api:gax-httpjson:0.52.0:default
      com.google.api-client:google-api-client:1.27.0:default
      com.google.api-client:google-api-client-jackson2:1.27.0:default
      com.google.api-client:google-api-client-java6:1.27.0:default
      com.google.api.grpc:grpc-google-cloud-bigquerystorage-v1beta1:0.44.0:default
      com.google.api.grpc:grpc-google-cloud-bigtable-admin-v2:0.38.0:default
      com.google.api.grpc:grpc-google-cloud-bigtable-v2:0.38.0:default
      com.google.api.grpc:grpc-google-cloud-pubsub-v1:1.43.0:default
      com.google.api.grpc:grpc-google-cloud-spanner-admin-database-v1:1.6.0:default
      com.google.api.grpc:grpc-google-cloud-spanner-admin-instance-v1:1.6.0:default
      com.google.api.grpc:grpc-google-cloud-spanner-v1:1.6.0:default
      com.google.api.grpc:grpc-google-common-protos:1.12.0:default
      com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1:0.44.0:default
      com.google.api.grpc:proto-google-cloud-bigtable-admin-v2:0.38.0:default
      com.google.api.grpc:proto-google-cloud-bigtable-v2:0.44.0:default
      com.google.api.grpc:proto-google-cloud-datastore-v1:0.44.0:default
      com.google.api.grpc:proto-google-cloud-pubsub-v1:1.43.0:default
      com.google.api.grpc:proto-google-cloud-spanner-admin-database-v1:1.6.0:default
      com.google.api.grpc:proto-google-cloud-spanner-admin-instance-v1:1.6.0:default
      com.google.api.grpc:proto-google-cloud-spanner-v1:1.6.0:default
      com.google.api.grpc:proto-google-common-protos:1.12.0:default
      com.google.api.grpc:proto-google-iam-v1:0.12.0:default
      com.google.apis:google-api-services-bigquery:v2-rev20181104-1.27.0:default
      com.google.apis:google-api-services-cloudresourcemanager:v1-rev20181015-1.27.0:default
      com.google.apis:google-api-services-pubsub:v1-rev20181105-1.27.0:default
      com.google.apis:google-api-services-storage:v1-rev20181109-1.27.0:default
      com.google.auth:google-auth-library-credentials:0.12.0:default
      com.google.auth:google-auth-library-oauth2-http:0.12.0:default
      com.google.auto.value:auto-value-annotations:1.6.3:default
      com.google.cloud:google-cloud-bigquerystorage:0.79.0-alpha:default
      com.google.cloud:google-cloud-bigtable:0.73.0-alpha:default
      com.google.cloud:google-cloud-bigtable-admin:0.73.0-alpha:default
      com.google.cloud:google-cloud-core:1.61.0:default
      com.google.cloud:google-cloud-core-grpc:1.61.0:default
      com.google.cloud:google-cloud-core-http:1.55.0:default
      com.google.cloud:google-cloud-spanner:1.6.0:default
      com.google.cloud.bigdataoss:gcsio:1.9.16:default
      com.google.cloud.bigdataoss:util:1.9.16:default
      com.google.cloud.bigtable:bigtable-client-core:1.8.0:default
      com.google.cloud.datastore:datastore-v1-proto-client:1.6.0:default
      com.google.code.findbugs:jsr305:3.0.2:default
      com.google.code.gson:gson:2.7:default
      com.google.errorprone:error_prone_annotations:2.3.2:default
      com.google.flogger:flogger:0.3.1:default
      com.google.flogger:flogger-system-backend:0.3.1:default
      com.google.flogger:google-extensions:0.3.1:default
      com.google.guava:failureaccess:1.0.1:default
      com.google.guava:guava:27.0.1-jre:default
      com.google.guava:listenablefuture:9999.0-empty-to-avoid-conflict-with-guava:default
      com.google.http-client:google-http-client:1.27.0:default
      com.google.http-client:google-http-client-appengine:1.27.0:default
      com.google.http-client:google-http-client-jackson:1.20.0:default
      com.google.http-client:google-http-client-jackson2:1.27.0:default
      com.google.http-client:google-http-client-protobuf:1.20.0:default
      com.google.j2objc:j2objc-annotations:1.1:default
      com.google.oauth-client:google-oauth-client:1.27.0:default
      com.google.oauth-client:google-oauth-client-java6:1.27.0:default
      com.google.protobuf:protobuf-java:3.6.1:default
      com.google.protobuf:protobuf-java-util:3.6.1:default
      com.google.protobuf.nano:protobuf-javanano:3.0.0-alpha-5:default
      com.squareup.okhttp:okhttp:2.5.0:default
      com.squareup.okio:okio:1.13.0:default
      com.thoughtworks.paranamer:paranamer:2.7:default
      commons-codec:commons-codec:1.10:default
      commons-logging:commons-logging:1.2:default
      io.dropwizard.metrics:metrics-core:3.1.2:default
      io.grpc:grpc-all:1.17.1:default
      io.grpc:grpc-alts:1.17.1:default
      io.grpc:grpc-auth:1.17.1:default
      io.grpc:grpc-context:1.17.1:default
      io.grpc:grpc-core:1.17.1:default
      io.grpc:grpc-grpclb:1.17.1:default
      io.grpc:grpc-netty:1.17.1:default
      io.grpc:grpc-netty-shaded:1.17.1:default
      io.grpc:grpc-okhttp:1.17.1:default
      io.grpc:grpc-protobuf:1.17.1:default
      io.grpc:grpc-protobuf-lite:1.17.1:default
      io.grpc:grpc-protobuf-nano:1.17.1:default
      io.grpc:grpc-stub:1.17.1:default
      io.grpc:grpc-testing:1.17.1:default
      io.netty:netty-buffer:4.1.30.Final:default
      io.netty:netty-codec:4.1.30.Final:default
      io.netty:netty-codec-http:4.1.30.Final:default
      io.netty:netty-codec-http2:4.1.30.Final:default
      io.netty:netty-codec-socks:4.1.30.Final:default
      io.netty:netty-common:4.1.30.Final:default
      io.netty:netty-handler:4.1.30.Final:default
      io.netty:netty-handler-proxy:4.1.30.Final:default
      io.netty:netty-resolver:4.1.30.Final:default
      io.netty:netty-tcnative-boringssl-static:2.0.17.Final:default
      io.netty:netty-transport:4.1.30.Final:default
      io.opencensus:opencensus-api:0.17.0:default
      io.opencensus:opencensus-contrib-grpc-metrics:0.17.0:default
      io.opencensus:opencensus-contrib-grpc-util:0.17.0:default
      io.opencensus:opencensus-contrib-http-util:0.15.0:default
      javax.annotation:javax.annotation-api:1.2:default
      joda-time:joda-time:2.10.3:default
      junit:junit:4.12:default
      org.apache.avro:avro:1.8.2:default
      org.apache.beam:beam-model-job-management:2.16.0:default
      org.apache.beam:beam-model-pipeline:2.16.0:default
      org.apache.beam:beam-sdks-java-core:2.16.0:default
      org.apache.beam:beam-sdks-java-extensions-google-cloud-platform-core:2.16.0:default
      org.apache.beam:beam-sdks-java-extensions-protobuf:2.16.0:default
      org.apache.beam:beam-sdks-java-io-google-cloud-platform:2.16.0:default
      org.apache.beam:beam-vendor-bytebuddy-1_9_3:0.1:default
      org.apache.beam:beam-vendor-grpc-1_21_0:0.1:default
      org.apache.beam:beam-vendor-guava-26_0-jre:0.1:default
      org.apache.commons:commons-compress:1.8.1:default
      org.apache.commons:commons-lang3:3.5:default
      org.apache.httpcomponents:httpclient:4.5.5:default
      org.apache.httpcomponents:httpcore:4.4.9:default
      org.checkerframework:checker-qual:2.5.2:default
      org.codehaus.jackson:jackson-core-asl:1.9.13:default
      org.codehaus.jackson:jackson-mapper-asl:1.9.13:default
      org.codehaus.mojo:animal-sniffer-annotations:1.17:default
      org.hamcrest:hamcrest-core:1.3:default
      org.mockito:mockito-core:1.9.5:default
      org.objenesis:objenesis:1.0:default
      org.slf4j:slf4j-api:1.7.25:default
      org.threeten:threetenbp:1.3.3:default
      org.tukaani:xz:1.8:default
      org.xerial.snappy:snappy-java:1.1.4:default

      Attachments

        Activity

          People

            Unassigned Unassigned
            jto Julien Tournay

            Dates

              Created:
              Updated:

              Slack

                Issue deployment