Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-14248

Retire SharedInstanceProfileCredentialsProvider in trunk.

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0-alpha4
    • 3.0.0-alpha4
    • fs/s3
    • None
    • Incompatible change, Reviewed
    • SharedInstanceProfileCredentialsProvider is removed after this change. Users should use InstanceProfileCredentialsProvider provided by AWS SDK instead, which itself enforces a singleton instance to reduce calls to AWS EC2 Instance Metadata Service.

    Description

      This is from the discussion in HADOOP-13050.

      So HADOOP-13727 added the SharedInstanceProfileCredentialsProvider, which effectively reduces high number of connections to EC2 Instance Metadata Service caused by InstanceProfileCredentialsProvider. That patch, in order to prevent the throttling problem, defined new class SharedInstanceProfileCredentialsProvider as a subclass of InstanceProfileCredentialsProvider, which enforces creation of only a single instance.

      Per HADOOP-13050, we upgraded the AWS Java SDK. Since then, the InstanceProfileCredentialsProvider in SDK code internally enforces a singleton. That confirms that our effort in HADOOP-13727 makes 100% sense. Meanwhile, SharedInstanceProfileCredentialsProvider can retire gracefully in trunk branch.

      Attachments

        1. HADOOP-14248.000.patch
          12 kB
          Mingliang Liu
        2. HADOOP-14248.001.patch
          12 kB
          Mingliang Liu
        3. HADOOP-14248-branch-2.001.patch
          11 kB
          Mingliang Liu
        4. HADOOP-14248-branch-2.002.patch
          11 kB
          Mingliang Liu

        Issue Links

          Activity

            People

              liuml07 Mingliang Liu
              liuml07 Mingliang Liu
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: