Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-14390

Kafka and Zookeeper with FIPS SASL has error

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.7.1, 3.3.1
    • None
    • security
    • None

    Description

      When setting Zookeeper to work with SASL plain and Kafka to connect to zookeeper with SASL and Java has FIPS enabled on a FIPS enabled Redhat machine
      Kafka cannot connect with zookeeper

      Used Apache Kafka 3.3.1 for both zookeeper and client
      Operating system Redhat 8
      Java 11 installed using yum install java-11-openjdk
      set fips mode using command as root
      fips-mode-setup --enable
      rebooted machine using
      systemctl reboot
      verified fips mode with
      fips-mode-setup --check

      set JAVA_HOME to /usr/lib/jvm/jre-11

      in config/zookeeper.properties, added properties
      admin.enableServer = false
      authProvider.1 = org.apache.zookeeper.server.auth.SASLAuthenticationProvider
      requireClientAuthScheme = sasl

      in config/server.properties, added properties
      advertised.listeners = SASL_PLAINTEXT://localhost:9092
      listeners = SASL_PLAINTEXT://:9092
      security.inter.broker.protocol = SASL_PLAINTEXT
      sasl.mechanism.inter.broker.protocol = PLAIN
      sasl.enabled.mechanisms = PLAIN
      super.users = User:admin

      $HOME/zookeeper_jaas.conf value:
      Server

      {     org.apache.kafka.common.security.plain.PlainLoginModule required     username="admin"     password="0x572f372b5da34874a3fdf4d8002f"     user_admin="0x572f372b5da34874a3fdf4d8002f"; };

      $HOME/server_jaas.conf value:
      KafkaServer {     org.apache.kafka.common.security.plain.PlainLoginModule required     username="admin"     password="0x572f372b5da34874a3fdf4d8002f"     user_admin="0x572f372b5da34874a3fdf4d8002f"; }

      ;
      Client

      {     org.apache.kafka.common.security.plain.PlainLoginModule required     username="admin"     password="0x572f372b5da34874a3fdf4d8002f"; }

      ;

      for zookeeper, in kafka directory ran
      setenv JAVA_HOME /usr/lib/jvm/jre-11
      setenv KAFKA_OPTS -Djava.security.auth.login.config=$HOME/zookeeper_jaas.conf
      ./bin/zookeeper-server-start.sh ./config/zookeeper.properties

      for kafka, in kafka directory ran
      setenv JAVA_HOME /usr/lib/jvm/jre-11
      setenv KAFKA_OPTS -Djava.security.auth.login.config=$HOME/server_jaas.conf
      ./bin/kafka-server-start.sh ./config/server.properties

      Kafka exits with SASL error
      [2022-11-15 13:21:04,344] ERROR SASL authentication with Zookeeper Quorum member failed. (org.apache.zookeeper.ClientCnxn)
      javax.security.sasl.SaslException: saslClient failed to initialize properly: it's null.
              at org.apache.zookeeper.client.ZooKeeperSaslClient.initialize(ZooKeeperSaslClient.java:399)
              at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1220)
      [2022-11-15 13:21:04,348] INFO EventThread shut down for session: 0x1000575ab110000 (org.apache.zookeeper.ClientCnxn)
      [2022-11-15 13:21:04,348] INFO [ZooKeeperClient Kafka server] Closing. (kafka.zookeeper.ZooKeeperClient)
      [2022-11-15 13:21:04,351] INFO [ZooKeeperClient Kafka server] Closed. (kafka.zookeeper.ZooKeeperClient)
      [2022-11-15 13:21:04,355] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
      kafka.zookeeper.ZooKeeperClientAuthFailedException: Auth failed either before or while waiting for connection
              at kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:260)
              at kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:108)
              at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1980)
              at kafka.server.KafkaServer.initZkClient(KafkaServer.scala:503)
              at kafka.server.KafkaServer.startup(KafkaServer.scala:203)
              at kafka.Kafka$.main(Kafka.scala:109)
              at kafka.Kafka.main(Kafka.scala)
      [2022-11-15 13:21:04,356] INFO shutting down (kafka.server.KafkaServer)
      [2022-11-15 13:21:04,363] INFO App info kafka.server for 0 unregistered (org.apache.kafka.common.utils.AppInfoParser)
      [2022-11-15 13:21:04,363] INFO shut down completed (kafka.server.KafkaServer)
      [2022-11-15 13:21:04,363] ERROR Exiting Kafka due to fatal exception during startup. (kafka.Kafka$)
      kafka.zookeeper.ZooKeeperClientAuthFailedException: Auth failed either before or while waiting for connection
              at kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:260)
              at kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:108)
              at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1980)
              at kafka.server.KafkaServer.initZkClient(KafkaServer.scala:503)
              at kafka.server.KafkaServer.startup(KafkaServer.scala:203)
              at kafka.Kafka$.main(Kafka.scala:109)
              at kafka.Kafka.main(Kafka.scala)
      [2022-11-15 13:21:04,368] INFO shutting down (kafka.server.KafkaServer)

      When removing FIPS by changing the java conf/security/java.security to have
      security.useSystemPropertiesFile=false

      The problem does not happen but this property disables FIPS

      Attachments

        Activity

          People

            Unassigned Unassigned
            lkgen lkgen
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: