Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-18073 S3A: Upgrade AWS SDK to V2
  3. HADOOP-18742

AWS v2 SDK: stabilise dependencies with rest of hadoop libraries

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Blocker
    • Resolution: Not A Problem
    • None
    • None
    • build, fs/s3

    Description

      aws v2 sdk dependencies need to

      1. be in sync with rest of hadoop
      2. not include anything coming in from hadoop common
      3. not export in hadoop-cloud-storage stuff from hadoop-common.

      currently it is pulling in a version of jackson cbor whose consistency with hadoop's import is simply luck, joda time is also there

      In an ideal world all this should be shaded: we cannot have the AWS sdk dictate what jackson version we ship with, given the history of downstream problems there.

      [INFO] +- com.amazonaws:aws-java-sdk-core:jar:1.12.316:compile
      [INFO] |  +- software.amazon.ion:ion-java:jar:1.0.2:compile
      [INFO] |  +- com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:jar:2.12.7:compile
      [INFO] |  \- joda-time:joda-time:jar:2.8.1:compile
      [INFO] +- software.amazon.awssdk:bundle:jar:2.19.12:compile
      [INFO] |  \- software.amazon.eventstream:eventstream:jar:1.0.1:compile
      [INFO] +- software.amazon.awssdk.crt:aws-crt:jar:0.21.0:compile
      
      

      Attachments

        Issue Links

          Activity

            People

              stevel@apache.org Steve Loughran
              stevel@apache.org Steve Loughran
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: