Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-18477 Über-jira: S3A Hadoop 3.3.9 features
  3. HADOOP-16806

AWS AssumedRoleCredentialProvider needs ExternalId add

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • 3.2.1
    • None
    • fs/s3
    • Patch

    Description

      AWS has added a security feature to the assume-role function in the form of the "ExternalId" key in the AWS Java SDK STSAssumeRoleSessionCredentialsProvider.Builder class.  To support this security feature, the hadoop aws AssumedRoleCredentialProvider needs a patch to include this value from the configuration as well as an added Constant to the org.apache.hadoop.fs.s3a.Constants file.

      The ExternalId is not a required security feature, it is an augmentation of the current assume role configuration. 

      Proposed: 

      • Get the assume-role ExternalId token from the configuration for the configuration key fs.s3a.assumed.role.externalid
      • Use the configured ExternalId value in the STSAssumeRoleSessionCredentialsProvider.Builder   

      e.g.

      if (StringUtils.isNotEmpty(externalId)) {
          builder.withExternalId(externalId); // include the token for cross-account assume role
      }

       Tests:

      • Unit test which verifies the ExternalId state value of the AssumedRoleCredentialProvider is consistent with the configured value - either empty or populated
      • Question: not sure about how to write the integration test for this feature.  We have an account configured for this use-case that verifies this feature but I don't have much context on the Hadoop project AWS S3 integration tests, perhaps a pointer could help.

       

       

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              jhartlaub Jon Hartlaub
              Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated: