Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12807

S3AFileSystem should read AWS credentials from environment variables

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.7.2
    • 2.8.0, 3.0.0-alpha1
    • fs/s3
    • None
    • Adds support to S3AFileSystem for reading AWS credentials from environment variables.

    Description

      Unlike the DefaultAWSCredentialsProviderChain in the AWS SDK, the AWSCredentialsProviderChain constructed by S3AFileSystem does not include an EnvironmentVariableCredentialsProvider instance. This prevents users from supplying AWS credentials in the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, which is the only alternative in some scenarios.

      In my scenario, I need to access S3 from within a test running in a CI environment that does not support IAM roles but does allow me to supply encrypted environment variables. Thus, the only secure approach I can use is to supply my AWS credentials in environment variables (plaintext configuration files are out of the question).

      Attachments

        1. HADOOP-12807-1.patch
          1 kB
          Tobin Baker
        2. HADOOP-12807-branch-2-004.patch
          3 kB
          Steve Loughran

        Issue Links

          Activity

            People

              tdbaker Tobin Baker
              tdbaker Tobin Baker
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: