XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • 3.3.0
    • 3.3.0, 3.2.2, 3.4.0
    • capacityscheduler
    • None
    • Reviewed

    Description

      YARN RMWebServices /scheduler-conf/validate leaks ZK Connections. Validation API creates a new CapacityScheduler and missed to close after the validation. Every CapacityScheduler#init opens MutableCSConfigurationProvider which opens ZKConfigurationStore and creates a ZK Connection.

      ZK LOGS

      -03-12 16:45:51,881 WARN org.apache.zookeeper.server.NIOServerCnxnFactory: [2 times] Error accepting new connection: Too many connections from /172.27.99.64 - max is 60
      2020-03-12 16:45:52,449 WARN org.apache.zookeeper.server.NIOServerCnxnFactory: Error accepting new connection: Too many connections from /172.27.99.64 - max is 60
      2020-03-12 16:45:52,710 WARN org.apache.zookeeper.server.NIOServerCnxnFactory: Error accepting new connection: Too many connections from /172.27.99.64 - max is 60
      2020-03-12 16:45:52,876 WARN org.apache.zookeeper.server.NIOServerCnxnFactory: [4 times] Error accepting new connection: Too many connections from /172.27.99.64 - max is 60
      2020-03-12 16:45:53,068 WARN org.apache.zookeeper.server.NIOServerCnxnFactory: [2 times] Error accepting new connection: Too many connections from /172.27.99.64 - max is 60
      2020-03-12 16:45:53,391 WARN org.apache.zookeeper.server.NIOServerCnxnFactory: [2 times] Error accepting new connection: Too many connections from /172.27.99.64 - max is 60
      2020-03-12 16:45:54,008 WARN org.apache.zookeeper.server.NIOServerCnxnFactory: Error accepting new connection: Too many connections from /172.27.99.64 - max is 60
      2020-03-12 16:45:54,287 WARN org.apache.zookeeper.server.NIOServerCnxnFactory: Error accepting new connection: Too many connections from /172.27.99.64 - max is 60
      2020-03-12 16:45:54,483 WARN org.apache.zookeeper.server.NIOServerCnxnFactory: [4 times] Error accepting new connection: Too many connections from /172.27.99.64 - max is 60
      

      And there is an another bug in ZKConfigurationStore which has not handled close() of ZKCuratorManager.

      Attachments

        1. YARN-10194-001.patch
          7 kB
          Prabhu Joseph
        2. YARN-10194-002.patch
          7 kB
          Prabhu Joseph
        3. YARN-10194-003.patch
          4 kB
          Prabhu Joseph
        4. YARN-10194-004.patch
          4 kB
          Prabhu Joseph
        5. YARN-10194-005.patch
          7 kB
          Prabhu Joseph
        6. YARN-10194-branch-3.2.001.patch
          7 kB
          Prabhu Joseph

        Activity

          People

            prabhujoseph Prabhu Joseph
            akhilpb Akhil PB
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: