Uploaded image for project: 'Livy'
  1. Livy
  2. LIVY-44

add Livy parameters --principal and --keytab parameters to be passed to spark-submit

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Won't Fix
    • 0.2
    • None
    • Core
    • kerberized hadoop

    Description

      It seems the only correct way to run Livy server is to run it under hue user in a kerberized cluster?
      There are two advantages:
      1. hue user is normally (by default in CDH) is allowed to impersonate to other users;
      2. hue already has a keytab maintened by Cloduera Manager.

      This is explained in http://www.cloudera.com/content/www/en-us/documentation/enterprise/latest/topics/cm_sg_yarn_long_jobs.html
      "Configuring Spark on YARN for Long-running Applications".
      Assuming Livy Server creates long-running Spark applications.

      Without --principal and --keytab parameters (and without non-expired kerberos ticket), I am getting:

      The Spark session could not be created in the cluster: ... 37 more
      Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
      at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
      at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:121)
      at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
      at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:223)
      at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
      at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
      at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193) ...
      46 more
      Without --principal and --keytab parameters, but with a non-expired kerberos ticket for my personal principal, I am getting:

      16/01/16 22:59:00 INFO RMProxy:
      Connecting to ResourceManager at xx.yy.com/10.20.xx.yy:8032
      16/01/16 22:59:00 WARN UserGroupInformation:
      PriviledgedActionException as:rdautkha (auth:PROXY)
      via rdautkhanov@XX.COM (auth:KERBEROS)
      cause:org.apache.hadoop.security.authorize.AuthorizationException:
      User: rdautkhanov@XX.COM is not allowed to impersonate rdautkha
      ps. hue.keytab is located in

      $ sudo ls -ltr /var/run/cloudera-scm-agent/process | grep hue-KT_RENEWER | tail -1 | awk '

      {print $9}

      '
      on hosts running "Kerberos Ticket Renewer" Hue role. This is the directory that must be passed to --keytab parameter to spark-submit.

      Irrespective of Hue, Livy Server should have parameters to allow to pass --principal and --keytab to spark-submit. So it's easier (or maybe the only way) to use in kerberized environmens.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              Tagar Ruslan Dautkhanov
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: