Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11151

Automatically refresh auth token and retry on auth failure

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.6.0
    • 2.6.0
    • security
    • None

    Description

      Enable CFS and KMS service in the cluster, initially it worked to put/copy file into encryption zone. But after a while (might be one day), it fails to put/copy file into the encryption zone with the error
      java.util.concurrent.ExecutionException: java.io.IOException: HTTP status [403], message [Forbidden]

      The kms.log shows below
      AbstractDelegationTokenSecretManager - Updating the current master key for generating delegation tokens
      2014-09-29 13:18:46,599 WARN AuthenticationFilter - AuthenticationToken ignored: org.apache.hadoop.security.authentication.util.SignerException: Invalid signature
      2014-09-29 13:18:46,599 WARN AuthenticationFilter - Authentication exception: Anonymous requests are disallowed
      org.apache.hadoop.security.authentication.client.AuthenticationException: Anonymous requests are disallowed
      at org.apache.hadoop.security.authentication.server.PseudoAuthenticationHandler.authenticate(PseudoAuthenticationHandler.java:184)
      at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:331)
      at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:507)
      at org.apache.hadoop.crypto.key.kms.server.KMSAuthenticationFilter.doFilter(KMSAuthenticationFilter.java:129)
      at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
      at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
      at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
      at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
      at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
      at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
      at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
      at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
      at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861)
      at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:606)
      at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
      at java.lang.Thread.run(Thread.java:745)

      Attachments

        1. HADOOP-11151.5.patch
          14 kB
          Arun Suresh
        2. HADOOP-11151.4.patch
          13 kB
          Arun Suresh
        3. HADOOP-11151.3.patch
          13 kB
          Arun Suresh
        4. HADOOP-11151.2.patch
          11 kB
          Arun Suresh
        5. HADOOP-11151.1.patch
          11 kB
          Arun Suresh

        Issue Links

          Activity

            People

              asuresh Arun Suresh
              zb161 zhubin
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: