Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-9708 Yarn Router Support DelegationToken
  3. YARN-11448

[Federation] Make Router Delegation token secret manager completely stateless

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.4.0
    • None
    • router

    Description

      Currently router secret manager requires routers to be stateful & with clients using sticky sessions.

      Otherwise, there are several issues mentioned below which lead to the delegation token functionality not working across router instances

      Eg:

      1. allKeys needs to be consistently updated across all router instances
      2. DB update exceptions are swallowed & returned as a success if just in memory variables are updated
      3. Purging Delegation Token / Master key on expiry assumes all tokens are available in memory
      4. APIs like get all tokens return only in memory data which is incorrect

      A more scalable & maintainable framework for Router would be to be design it as a stateless service. Given database KV lookups are in the order of < 10 ms, it doesn't add any latency overhead and makes router easier to maintain. Plus a stateless router setup, with no assumptions of stickiness makes the router framework more generic. 

      Additionally, some of the functionality around master key ids, delegation token sequence numbers is implemented as globally autoincrement ids which too isn't feasible across all datastores. The actual requirement is to generate unique keys for master key ids / delegation tokens which is a much more simpler & generic solution. Plus certain apis like get sequence no / set sequence no aren't applicable for router and we can avoid providing them to make things much more simpler.  

      This patch addresses these functional concerns while working within the interfaces of AbstractDelegationTokenSecretManager. 

      As a later patch, we can create better delegation token interfaces to support both stateful & stateless secret managers. 

      Attachments

        Issue Links

          Activity

            People

              krishan1390 Krishan Goyal
              krishan1390 Krishan Goyal
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated: