Uploaded image for project: 'ZooKeeper'
  1. ZooKeeper
  2. ZOOKEEPER-2785

Server inappropriately throttles connections under load before SASL completes

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 3.4.10
    • 3.4.11, 3.5.4, 3.6.0
    • server

    Description

      When a zk server is running close to its outstanding requests limit, the server incorrectly throttles the sasl request. This leads to the client waiting for the final sasl packet (session is already established) and deferring all non priming packets till then which also includes the ping packets. The client then waits for the final packet but never gets it and times out saying haven't heard from server. This is fatal for services such as HBase which retry for finite attempts and exit post these attempts.

      Issue being that in ZooKeeperServer.processPacket(..) incase of sasl we send the response and incorrectly also call cnxn.incrOutstandingRequests(h), which throttles the connection if we're running over outstandingrequests limit, which results in the server not processing the subsequent packet from the client. Also we donot have any pending request to send for the connection and hence never call enableRecv(). We should return after sending response to the sasl request.

      Attachments

        Issue Links

          Activity

            People

              abhishek.chouhan Abhishek Singh Chouhan
              abhishek.chouhan Abhishek Singh Chouhan
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: