Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-19143

API in Spark for distributing new delegation tokens (Improve delegation token handling in secure clusters)

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Incomplete
    • 2.0.2, 2.1.0
    • None
    • Spark Core, YARN

    Description

      Spin off from SPARK-14743 and comments chain in recent comments in SPARK-5493.

      Spark currently doesn't have a way for distribution new delegation tokens. Quoting vanzin from SPARK-5493

      IIRC Livy doesn't yet support delegation token renewal. Once it reaches the TTL, the session is unusable.
      There might be ways to hack support for that without changes in Spark, but I'd like to see a proper API in Spark for distributing new delegation tokens. I mentioned that in SPARK-14743, but although that bug is closed, that particular feature hasn't been implemented yet.

      Other thoughts?

      Attachments

        Issue Links

          causes

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-15570 renewable delegation tokens for long-lived spark applications

          • Major - Major loss of function.
          • Resolved
          is part of

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-14743 Improve delegation token handling in secure clusters

          • Major - Major loss of function.
          • Resolved
          is related to

          New Feature - A new feature of the product, which has yet to be developed. SPARK-16742 Kerberos support for Spark on Mesos

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. ZEPPELIN-1730 impersonate spark interpreter using --proxy-user

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-21737 Create communication channel between arbitrary clients and the Spark AM in YARN mode

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-5493 Support proxy users under kerberos

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link Design doc for SPARK-14743

          Web Link LIVY-296

          Activity

            People

              Unassigned Unassigned
              Tagar Ruslan Dautkhanov
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: