Description
Although i'm able to authenticate successfully with the kerberoes account "zookeeper/kafka-d1.eng.company.com@COMPANY.COM" , i still happen to encounter AUTH_FAILED during client Authentication
Following is the verification made from my end :
- Checked DNS ( Both Forward and Backward)
nslookup kafka-d1.eng.company.com
Server: 172.16.2.3
Address: 172.16.2.3#53
Name: kafka-d1.eng.company.com
Address: 10.14.61.17
Reverse DNS
nslookup 10.14.61.17
Server: 172.16.2.3
Address: 172.16.2.3#53
17.61.14.10.in-addr.arpa name = kafka-d1.eng.company.com.
2. Kerberoes Authentication
kinit -kt /etc/keytabs/zookeeper.keytab -V zookeeper/kafka-d1.eng.company.com
Using default cache: /tmp/krb5cc_0
Using principal: zookeeper/kafka-d1.eng.company.com@COMPANY.COM
Using keytab: /etc/keytabs/zookeeper.keytab
Authenticated to Kerberos v5
Below is the krb5 configuration File:
cat /etc/krb5.conf
[libdefaults]
default_realm = COMPANY.COM
dns_lookup_kdc = true
dns_lookup_realm = true
ticket_lifetime = 86400
renew_lifetime = 604800
forwardable = true
default_tgs_enctypes = aes256-cts
default_tkt_enctypes = aes256-cts
permitted_enctypes = aes256-cts
udp_preference_limit = 1
kdc_timeout = 3000
ignore_acceptor_hostname = true
[realms]
COMPANY.COM =
[domain_realm]
kafka-d1.eng.company.com = COMPANY.COM
export JVMFLAGS=-Djava.security.auth.login.config=/usr/share/zookeeper/conf/client_jaas.conf -Dsun.security.krb5.debug=true
cat /usr/share/zookeeper/conf/client_jaas.conf
Client
;
Error Message :zoo.cfgzookeeper_server.log
./zkCli.sh -server kafka-d1.eng.company.com:2181 Connecting to kafka-d1.eng.company.com:2181 2019-10-14 02:08:16,625 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT 2019-10-14 02:08:16,628 [myid:] - INFO [main:Environment@100] - Client environment:host.name=kafka-d1.eng.company.com 2019-10-14 02:08:16,628 [myid:] - INFO [main:Environment@100] - Client environment:java.version=1.8.0_201 2019-10-14 02:08:16,630 [myid:] - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation 2019-10-14 02:08:16,630 [myid:] - INFO [main:Environment@100] - Client environment:java.home=/opt/jdk1.8.0_201/jre 2019-10-14 02:08:16,630 [myid:] - INFO [main:Environment@100] - Client environment:java.class.path=/usr/share/zookeeper/bin/../build/classes:/usr/share/zookeeper/bin/../build/lib/*.jar:/usr/share/zookeeper/bin/../lib/slf4j-log4j12-1.6.1.jar:/usr/share/zookeeper/bin/../lib/slf4j-api-1.6.1.jar:/usr/share/zookeeper/bin/../lib/netty-3.10.5.Final.jar:/usr/share/zookeeper/bin/../lib/log4j-1.2.16.jar:/usr/share/zookeeper/bin/../lib/jline-0.9.94.jar:/usr/share/zookeeper/bin/../zookeeper-3.4.10.jar:/usr/share/zookeeper/bin/../src/java/lib/*.jar:/usr/share/zookeeper/bin/../conf: 2019-10-14 02:08:16,630 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2019-10-14 02:08:16,631 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp 2019-10-14 02:08:16,631 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA> 2019-10-14 02:08:16,631 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux 2019-10-14 02:08:16,631 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64 2019-10-14 02:08:16,631 [myid:] - INFO [main:Environment@100] - Client environment:os.version=3.10.0-327.el7.x86_64 2019-10-14 02:08:16,631 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root 2019-10-14 02:08:16,631 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/root 2019-10-14 02:08:16,631 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/usr/share/zookeeper-3.4.10/bin 2019-10-14 02:08:16,632 [myid:] - INFO [main:ZooKeeper@438] - Initiating client connection, connectString=kafka-d1.eng.company.com:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@306a30c7 Welcome to ZooKeeper! JLine support is enabled Debug is true storeKey true useTicketCache false useKeyTab true doNotPrompt false ticketCache is null isInitiator true KeyTab is /etc/keytabs/zookeeper.keytab refreshKrb5Config is false principal is zookeeper/kafka-d1.eng.company.com@COMPANY.COM tryFirstPass is false useFirstPass is false storePass is false clearPass is false [zk: kafka-d1.eng.company.com:2181(CONNECTING) 0] principal is zookeeper/kafka-d1.eng.company.com@COMPANY.COM Will use keytab Commit Succeeded 2019-10-14 02:08:16,971 [myid:] - INFO [main-SendThread(kafka-d1.eng.company.com:2181):Login@295] - Client successfully logged in. 2019-10-14 02:08:16,973 [myid:] - INFO [Thread-1:Login$1@128] - TGT refresh thread started. 2019-10-14 02:08:16,975 [myid:] - INFO [Thread-1:Login@303] - TGT valid starting at: Mon Oct 14 02:08:16 EDT 2019 2019-10-14 02:08:16,976 [myid:] - INFO [Thread-1:Login@304] - TGT expires: Mon Oct 14 12:08:16 EDT 2019 2019-10-14 02:08:16,976 [myid:] - INFO [Thread-1:Login$1@183] - TGT refresh sleeping until: Mon Oct 14 10:08:57 EDT 2019 2019-10-14 02:08:16,977 [myid:] - INFO [main-SendThread(kafka-d1.eng.company.com:2181):SecurityUtils$1@124] - Client will use GSSAPI as SASL mechanism. 2019-10-14 02:08:16,988 [myid:] - INFO [main-SendThread(kafka-d1.eng.company.com:2181):ClientCnxn$SendThread@1032] - Opening socket connection to server kafka-d1.eng.company.com/10.14.61.17:2181. Will attempt to SASL-authenticate using Login Context section 'Client' 2019-10-14 02:08:16,994 [myid:] - INFO [main-SendThread(kafka-d1.eng.company.com:2181):ClientCnxn$SendThread@876] - Socket connection established to kafka-d1.eng.company.com/10.14.61.17:2181, initiating session 2019-10-14 02:08:17,002 [myid:] - INFO [main-SendThread(kafka-d1.eng.company.com:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server kafka-d1.eng.company.com/10.14.61.17:2181, sessionid = 0x16dc8cbdb3b0002, negotiated timeout = 30000WATCHER::WatchedEvent state:SyncConnected type:None path:null 2019-10-14 02:08:17,024 [myid:] - ERROR [main-SendThread(kafka-d1.eng.company.com:2181):ZooKeeperSaslClient@247] - SASL authentication failed using login context 'Client'.WATCHER::WatchedEvent state:AuthFailed type:None path:null