[ZOOKEEPER-2486] Flaky Test: org.apache.zookeeper.test.QuorumZxidSyncTest.testBehindLeader - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Sub-task
Status: Resolved
Priority: Major
Resolution: Cannot Reproduce
Affects Version/s: 3.4.8
Fix Version/s: 3.5.2
Component/s: tests
Labels:
- flaky
- flaky-test

Description

From https://builds.apache.org/job/ZooKeeper_branch34_jdk7/1156

Error Message

waiting for server up
Stacktrace

junit.framework.AssertionFailedError: waiting for server up
	at org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:183)
	at org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:113)
	at org.apache.zookeeper.test.QuorumZxidSyncTest.testBehindLeader(QuorumZxidSyncTest.java:67)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:55)
Standard Output

2016-07-21 08:11:45,722 [myid:] - INFO  [main:PortAssignment@32] - assigning port 11221
2016-07-21 08:11:45,729 [myid:] - INFO  [main:ZKTestCase$1@50] - STARTING testBehindLeader
2016-07-21 08:11:45,729 [myid:] - INFO  [main:QuorumBase@69] - QuorumBase.setup null
2016-07-21 08:11:45,926 [myid:] - INFO  [main:PortAssignment@32] - assigning port 11222
2016-07-21 08:11:45,926 [myid:] - INFO  [main:PortAssignment@32] - assigning port 11223
2016-07-21 08:11:45,927 [myid:] - INFO  [main:PortAssignment@32] - assigning port 11224
2016-07-21 08:11:45,927 [myid:] - INFO  [main:PortAssignment@32] - assigning port 11225
2016-07-21 08:11:45,927 [myid:] - INFO  [main:PortAssignment@32] - assigning port 11226
2016-07-21 08:11:45,928 [myid:] - INFO  [main:PortAssignment@32] - assigning port 11227
2016-07-21 08:11:45,928 [myid:] - INFO  [main:PortAssignment@32] - assigning port 11228
2016-07-21 08:11:45,928 [myid:] - INFO  [main:PortAssignment@32] - assigning port 11229
2016-07-21 08:11:45,928 [myid:] - INFO  [main:PortAssignment@32] - assigning port 11230
2016-07-21 08:11:45,929 [myid:] - INFO  [main:PortAssignment@32] - assigning port 11231
2016-07-21 08:11:45,929 [myid:] - INFO  [main:QuorumBase@93] - Ports are: 127.0.0.1:11222,127.0.0.1:11223,127.0.0.1:11224,127.0.0.1:11225,127.0.0.1:11226
2016-07-21 08:11:45,946 [myid:] - INFO  [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 127.0.0.1 to address: /127.0.0.1
2016-07-21 08:11:45,946 [myid:] - INFO  [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 127.0.0.1 to address: /127.0.0.1
2016-07-21 08:11:45,948 [myid:] - INFO  [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 127.0.0.1 to address: /127.0.0.1
2016-07-21 08:11:45,948 [myid:] - INFO  [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 127.0.0.1 to address: /127.0.0.1
2016-07-21 08:11:45,949 [myid:] - INFO  [main:QuorumPeer$QuorumServer@149] - Resolved hostname: 127.0.0.1 to address: /127.0.0.1
2016-07-21 08:11:45,949 [myid:] - INFO  [main:QuorumBase@142] - creating QuorumPeer 1 port 11222
2016-07-21 08:11:45,961 [myid:] - INFO  [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:11222
2016-07-21 08:11:45,979 [myid:] - INFO  [main:QuorumBase@145] - creating QuorumPeer 2 port 11223
2016-07-21 08:11:45,979 [myid:] - INFO  [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:11223
2016-07-21 08:11:45,980 [myid:] - INFO  [main:QuorumBase@148] - creating QuorumPeer 3 port 11224
2016-07-21 08:11:45,980 [myid:] - INFO  [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:11224
2016-07-21 08:11:45,981 [myid:] - INFO  [main:QuorumBase@151] - creating QuorumPeer 4 port 11225
2016-07-21 08:11:45,981 [myid:] - INFO  [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:11225
2016-07-21 08:11:45,981 [myid:] - INFO  [main:QuorumBase@154] - creating QuorumPeer 5 port 11226
2016-07-21 08:11:45,982 [myid:] - INFO  [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:11226
2016-07-21 08:11:45,982 [myid:] - INFO  [main:QuorumBase@163] - QuorumPeer 1 voting view: {1=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@47628981, 2=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@163198c4, 3=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@1224773e, 4=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@181090c0, 5=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@678f3997}
2016-07-21 08:11:45,983 [myid:] - INFO  [main:QuorumBase@164] - QuorumPeer 2 voting view: {1=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@47628981, 2=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@163198c4, 3=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@1224773e, 4=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@181090c0, 5=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@678f3997}
2016-07-21 08:11:45,983 [myid:] - INFO  [main:QuorumBase@165] - QuorumPeer 3 voting view: {1=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@47628981, 2=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@163198c4, 3=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@1224773e, 4=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@181090c0, 5=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@678f3997}
2016-07-21 08:11:45,983 [myid:] - INFO  [main:QuorumBase@166] - QuorumPeer 4 voting view: {1=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@47628981, 2=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@163198c4, 3=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@1224773e, 4=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@181090c0, 5=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@678f3997}
2016-07-21 08:11:45,983 [myid:] - INFO  [main:QuorumBase@167] - QuorumPeer 5 voting view: {1=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@47628981, 2=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@163198c4, 3=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@1224773e, 4=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@181090c0, 5=org.apache.zookeeper.server.quorum.QuorumPeer$QuorumServer@678f3997}
2016-07-21 08:11:45,983 [myid:] - INFO  [main:QuorumBase@169] - start QuorumPeer 1
2016-07-21 08:11:45,988 [myid:] - INFO  [main:QuorumPeer@533] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation
2016-07-21 08:11:45,990 [myid:] - INFO  [main:QuorumPeer@548] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation
2016-07-21 08:11:45,996 [myid:] - INFO  [ListenerThread:QuorumCnxManager$Listener@534] - My election bind port: /127.0.0.1:12227
2016-07-21 08:11:46,001 [myid:] - INFO  [main:QuorumBase@171] - start QuorumPeer 2
2016-07-21 08:11:46,002 [myid:] - INFO  [main:QuorumPeer@533] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation
2016-07-21 08:11:46,003 [myid:] - INFO  [main:QuorumPeer@548] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation
2016-07-21 08:11:46,004 [myid:] - INFO  [ListenerThread:QuorumCnxManager$Listener@534] - My election bind port: /127.0.0.1:12228
2016-07-21 08:11:46,006 [myid:] - INFO  [main:QuorumBase@173] - start QuorumPeer 3
2016-07-21 08:11:46,006 [myid:] - INFO  [main:QuorumPeer@533] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation
2016-07-21 08:11:46,007 [myid:] - INFO  [main:QuorumPeer@548] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation
2016-07-21 08:11:46,007 [myid:] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11222:QuorumPeer@774] - LOOKING
2016-07-21 08:11:46,008 [myid:] - INFO  [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11223:QuorumPeer@774] - LOOKING
2016-07-21 08:11:46,009 [myid:] - INFO  [ListenerThread:QuorumCnxManager$Listener@534] - My election bind port: /127.0.0.1:12229
2016-07-21 08:11:46,009 [myid:] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11222:FastLeaderElection@818] - New election. My id =  1, proposed zxid=0x0
2016-07-21 08:11:46,009 [myid:] - INFO  [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11223:FastLeaderElection@818] - New election. My id =  2, proposed zxid=0x0
2016-07-21 08:11:46,009 [myid:] - INFO  [main:QuorumBase@175] - start QuorumPeer 4
2016-07-21 08:11:46,010 [myid:] - INFO  [main:QuorumPeer@533] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation
2016-07-21 08:11:46,011 [myid:] - INFO  [main:QuorumPeer@548] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation
2016-07-21 08:11:46,011 [myid:] - INFO  [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:11224:QuorumPeer@774] - LOOKING
2016-07-21 08:11:46,011 [myid:] - INFO  [/127.0.0.1:12228:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:38924
2016-07-21 08:11:46,011 [myid:] - INFO  [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:11224:FastLeaderElection@818] - New election. My id =  3, proposed zxid=0x0
2016-07-21 08:11:46,011 [myid:] - INFO  [/127.0.0.1:12227:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:43415
2016-07-21 08:11:46,011 [myid:] - INFO  [WorkerSender[myid=1]:QuorumCnxManager@199] - Have smaller server identifier, so dropping the connection: (2, 1)
2016-07-21 08:11:46,011 [myid:] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,013 [myid:] - INFO  [WorkerSender[myid=1]:QuorumCnxManager@199] - Have smaller server identifier, so dropping the connection: (3, 1)
2016-07-21 08:11:46,014 [myid:] - INFO  [/127.0.0.1:12229:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:51548
2016-07-21 08:11:46,014 [myid:] - INFO  [main:QuorumBase@177] - start QuorumPeer 5
2016-07-21 08:11:46,015 [myid:] - INFO  [ListenerThread:QuorumCnxManager$Listener@534] - My election bind port: /127.0.0.1:12230
2016-07-21 08:11:46,015 [myid:] - INFO  [main:QuorumPeer@533] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation
2016-07-21 08:11:46,015 [myid:] - WARN  [WorkerSender[myid=1]:QuorumCnxManager@400] - Cannot open channel to 4 at election address /127.0.0.1:12230
java.net.ConnectException: Connection refused
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:579)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:381)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:354)
	at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:452)
	at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:433)
	at java.lang.Thread.run(Thread.java:745)
2016-07-21 08:11:46,018 [myid:] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,018 [myid:] - INFO  [/127.0.0.1:12229:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:51551
2016-07-21 08:11:46,016 [myid:] - INFO  [/127.0.0.1:12227:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:43417
2016-07-21 08:11:46,016 [myid:] - INFO  [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:QuorumPeer@774] - LOOKING
2016-07-21 08:11:46,016 [myid:] - INFO  [main:QuorumPeer@548] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation
2016-07-21 08:11:46,019 [myid:] - INFO  [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:FastLeaderElection@818] - New election. My id =  4, proposed zxid=0x0
2016-07-21 08:11:46,019 [myid:] - INFO  [/127.0.0.1:12230:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:46102
2016-07-21 08:11:46,018 [myid:] - INFO  [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,018 [myid:] - INFO  [WorkerSender[myid=2]:QuorumCnxManager@199] - Have smaller server identifier, so dropping the connection: (3, 2)
2016-07-21 08:11:46,018 [myid:] - INFO  [WorkerSender[myid=1]:QuorumPeer$QuorumServer@149] - Resolved hostname: 127.0.0.1 to address: /127.0.0.1
2016-07-21 08:11:46,018 [myid:] - INFO  [/127.0.0.1:12228:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:38928
2016-07-21 08:11:46,021 [myid:] - INFO  [WorkerSender[myid=2]:QuorumCnxManager@199] - Have smaller server identifier, so dropping the connection: (4, 2)
2016-07-21 08:11:46,021 [myid:] - INFO  [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,021 [myid:] - INFO  [/127.0.0.1:12229:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:51553
2016-07-21 08:11:46,020 [myid:] - INFO  [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 3 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,019 [myid:] - INFO  [WorkerSender[myid=3]:QuorumCnxManager@199] - Have smaller server identifier, so dropping the connection: (4, 3)
2016-07-21 08:11:46,022 [myid:] - INFO  [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,021 [myid:] - WARN  [WorkerSender[myid=2]:QuorumCnxManager@400] - Cannot open channel to 5 at election address /127.0.0.1:12231
java.net.ConnectException: Connection refused
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:579)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:381)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:354)
	at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:452)
	at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:433)
	at java.lang.Thread.run(Thread.java:745)
2016-07-21 08:11:46,021 [myid:] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 3 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,021 [myid:] - WARN  [WorkerSender[myid=1]:QuorumCnxManager@400] - Cannot open channel to 5 at election address /127.0.0.1:12231
java.net.ConnectException: Connection refused
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:579)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:381)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:354)
	at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:452)
	at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:433)
	at java.lang.Thread.run(Thread.java:745)
2016-07-21 08:11:46,023 [myid:] - INFO  [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 3 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,023 [myid:] - INFO  [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,023 [myid:] - INFO  [WorkerSender[myid=2]:QuorumPeer$QuorumServer@149] - Resolved hostname: 127.0.0.1 to address: /127.0.0.1
2016-07-21 08:11:46,024 [myid:] - INFO  [/127.0.0.1:12230:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:46104
2016-07-21 08:11:46,022 [myid:] - INFO  [ListenerThread:QuorumCnxManager$Listener@534] - My election bind port: /127.0.0.1:12231
2016-07-21 08:11:46,022 [myid:] - WARN  [WorkerSender[myid=3]:QuorumCnxManager@400] - Cannot open channel to 5 at election address /127.0.0.1:12231
java.net.ConnectException: Connection refused
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:579)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:381)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:354)
	at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:452)
	at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:433)
	at java.lang.Thread.run(Thread.java:745)
2016-07-21 08:11:46,025 [myid:] - INFO  [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 3 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,025 [myid:] - INFO  [WorkerReceiver[myid=4]:FastLeaderElection@600] - Notification: 1 (message format version), 3 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,025 [myid:] - INFO  [main:QuorumBase@179] - started QuorumPeer 5
2016-07-21 08:11:46,026 [myid:] - INFO  [main:QuorumBase@181] - Checking ports 127.0.0.1:11222,127.0.0.1:11223,127.0.0.1:11224,127.0.0.1:11225,127.0.0.1:11226
2016-07-21 08:11:46,025 [myid:] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 3 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,025 [myid:] - INFO  [/127.0.0.1:12227:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:43428
2016-07-21 08:11:46,027 [myid:] - INFO  [/127.0.0.1:12230:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:46109
2016-07-21 08:11:46,025 [myid:] - INFO  [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 3 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,024 [myid:] - INFO  [WorkerSender[myid=1]:QuorumPeer$QuorumServer@149] - Resolved hostname: 127.0.0.1 to address: /127.0.0.1
2016-07-21 08:11:46,027 [myid:] - INFO  [WorkerSender[myid=4]:QuorumCnxManager@199] - Have smaller server identifier, so dropping the connection: (5, 4)
2016-07-21 08:11:46,027 [myid:] - INFO  [WorkerReceiver[myid=4]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 4 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,026 [myid:] - INFO  [/127.0.0.1:12228:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:38938
2016-07-21 08:11:46,025 [myid:] - INFO  [WorkerSender[myid=3]:QuorumPeer$QuorumServer@149] - Resolved hostname: 127.0.0.1 to address: /127.0.0.1
2016-07-21 08:11:46,025 [myid:] - INFO  [WorkerSender[myid=2]:QuorumCnxManager@199] - Have smaller server identifier, so dropping the connection: (4, 2)
2016-07-21 08:11:46,028 [myid:] - INFO  [main:FourLetterWordMain@62] - connecting to 127.0.0.1 11222
2016-07-21 08:11:46,028 [myid:] - WARN  [RecvWorker:2:QuorumCnxManager$RecvWorker@810] - Connection broken for id 2, my id = 4, error = 
java.net.SocketException: Socket closed
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at java.net.SocketInputStream.read(SocketInputStream.java:210)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795)
2016-07-21 08:11:46,029 [myid:] - WARN  [RecvWorker:2:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker
2016-07-21 08:11:46,028 [myid:] - INFO  [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 4 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,027 [myid:] - INFO  [QuorumPeer[myid=5]/0:0:0:0:0:0:0:0:11226:QuorumPeer@774] - LOOKING
2016-07-21 08:11:46,027 [myid:] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,027 [myid:] - INFO  [/127.0.0.1:12231:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:52086
2016-07-21 08:11:46,030 [myid:] - WARN  [RecvWorker:4:QuorumCnxManager$RecvWorker@810] - Connection broken for id 4, my id = 2, error = 
java.io.EOFException
	at java.io.DataInputStream.readInt(DataInputStream.java:392)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795)
2016-07-21 08:11:46,030 [myid:] - WARN  [SendWorker:4:QuorumCnxManager$SendWorker@732] - Exception when using channel: for id 4 my id = 2 error = java.net.SocketException: Broken pipe
2016-07-21 08:11:46,029 [myid:] - INFO  [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,031 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11222:NIOServerCnxnFactory@192] - Accepted socket connection from /127.0.0.1:52363
2016-07-21 08:11:46,029 [myid:] - INFO  [/127.0.0.1:12228:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:38940
2016-07-21 08:11:46,029 [myid:] - INFO  [WorkerSender[myid=1]:QuorumCnxManager@199] - Have smaller server identifier, so dropping the connection: (5, 1)
2016-07-21 08:11:46,029 [myid:] - INFO  [WorkerReceiver[myid=4]:FastLeaderElection@600] - Notification: 1 (message format version), 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,032 [myid:] - INFO  [WorkerReceiver[myid=4]:FastLeaderElection@600] - Notification: 1 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,032 [myid:] - INFO  [WorkerReceiver[myid=4]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,029 [myid:] - WARN  [SendWorker:2:QuorumCnxManager$SendWorker@732] - Exception when using channel: for id 2 my id = 4 error = java.net.SocketException: Socket closed
2016-07-21 08:11:46,033 [myid:] - WARN  [SendWorker:2:QuorumCnxManager$SendWorker@736] - Send worker leaving thread
2016-07-21 08:11:46,029 [myid:] - INFO  [WorkerSender[myid=2]:QuorumCnxManager@199] - Have smaller server identifier, so dropping the connection: (5, 2)
2016-07-21 08:11:46,028 [myid:] - INFO  [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,033 [myid:] - INFO  [WorkerSender[myid=1]:QuorumCnxManager@199] - Have smaller server identifier, so dropping the connection: (5, 1)
2016-07-21 08:11:46,032 [myid:] - INFO  [WorkerReceiver[myid=4]:FastLeaderElection@600] - Notification: 1 (message format version), 3 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,031 [myid:] - INFO  [/127.0.0.1:12231:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:52088
2016-07-21 08:11:46,031 [myid:] - INFO  [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,031 [myid:] - INFO  [/127.0.0.1:12230:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:46117
2016-07-21 08:11:46,030 [myid:] - WARN  [SendWorker:4:QuorumCnxManager$SendWorker@736] - Send worker leaving thread
2016-07-21 08:11:46,030 [myid:] - WARN  [RecvWorker:4:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker
2016-07-21 08:11:46,030 [myid:] - INFO  [WorkerSender[myid=3]:QuorumCnxManager@199] - Have smaller server identifier, so dropping the connection: (5, 3)
2016-07-21 08:11:46,030 [myid:] - INFO  [QuorumPeer[myid=5]/0:0:0:0:0:0:0:0:11226:FastLeaderElection@818] - New election. My id =  5, proposed zxid=0x0
2016-07-21 08:11:46,030 [myid:] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 4 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,034 [myid:] - INFO  [/127.0.0.1:12228:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:38947
2016-07-21 08:11:46,034 [myid:] - INFO  [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 3 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,034 [myid:] - INFO  [WorkerReceiver[myid=4]:FastLeaderElection@600] - Notification: 1 (message format version), 3 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,033 [myid:] - INFO  [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,035 [myid:] - INFO  [/127.0.0.1:12231:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:52089
2016-07-21 08:11:46,035 [myid:] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,036 [myid:] - INFO  [WorkerReceiver[myid=4]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,036 [myid:] - INFO  [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 3 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,037 [myid:] - INFO  [/127.0.0.1:12227:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:43442
2016-07-21 08:11:46,036 [myid:] - INFO  [WorkerReceiver[myid=5]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 4 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,036 [myid:] - INFO  [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,037 [myid:] - INFO  [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 4 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,037 [myid:] - INFO  [WorkerReceiver[myid=4]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,038 [myid:] - INFO  [/127.0.0.1:12231:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:52091
2016-07-21 08:11:46,037 [myid:] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 3 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,036 [myid:] - INFO  [WorkerSender[myid=1]:QuorumCnxManager@199] - Have smaller server identifier, so dropping the connection: (5, 1)
2016-07-21 08:11:46,038 [myid:] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,038 [myid:] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,038 [myid:] - INFO  [WorkerReceiver[myid=5]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,038 [myid:] - INFO  [/127.0.0.1:12229:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:51573
2016-07-21 08:11:46,038 [myid:] - INFO  [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,037 [myid:] - INFO  [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,039 [myid:] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 5 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,039 [myid:] - INFO  [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,039 [myid:] - INFO  [WorkerReceiver[myid=5]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,040 [myid:] - INFO  [WorkerReceiver[myid=4]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 5 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,039 [myid:] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,039 [myid:] - INFO  [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 5 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,039 [myid:] - INFO  [/127.0.0.1:12231:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:52093
2016-07-21 08:11:46,040 [myid:] - INFO  [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,040 [myid:] - INFO  [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,040 [myid:] - INFO  [WorkerReceiver[myid=4]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,040 [myid:] - INFO  [WorkerReceiver[myid=5]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 5 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,041 [myid:] - INFO  [WorkerReceiver[myid=4]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,041 [myid:] - INFO  [/127.0.0.1:12227:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:43444
2016-07-21 08:11:46,041 [myid:] - INFO  [WorkerReceiver[myid=4]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 4 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,041 [myid:] - WARN  [RecvWorker:1:QuorumCnxManager$RecvWorker@810] - Connection broken for id 1, my id = 5, error = 
java.net.SocketException: Socket closed
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at java.io.DataInputStream.readFully(DataInputStream.java:195)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:805)
2016-07-21 08:11:46,042 [myid:] - WARN  [RecvWorker:1:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker
2016-07-21 08:11:46,041 [myid:] - INFO  [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 4 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,041 [myid:] - WARN  [SendWorker:1:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715)
2016-07-21 08:11:46,041 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11222:NIOServerCnxn@827] - Processing stat command from /127.0.0.1:52363
2016-07-21 08:11:46,041 [myid:] - WARN  [RecvWorker:5:QuorumCnxManager$RecvWorker@810] - Connection broken for id 5, my id = 1, error = 
java.io.EOFException
	at java.io.DataInputStream.readInt(DataInputStream.java:392)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795)
2016-07-21 08:11:46,043 [myid:] - WARN  [RecvWorker:5:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker
2016-07-21 08:11:46,041 [myid:] - INFO  [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,041 [myid:] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,043 [myid:] - INFO  [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 4 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,043 [myid:] - WARN  [SendWorker:1:QuorumCnxManager$SendWorker@736] - Send worker leaving thread
2016-07-21 08:11:46,042 [myid:] - INFO  [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 5 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,042 [myid:] - WARN  [SendWorker:5:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715)
2016-07-21 08:11:46,044 [myid:] - WARN  [SendWorker:5:QuorumCnxManager$SendWorker@736] - Send worker leaving thread
2016-07-21 08:11:46,042 [myid:] - INFO  [WorkerReceiver[myid=4]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,042 [myid:] - INFO  [/127.0.0.1:12231:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:52096
2016-07-21 08:11:46,041 [myid:] - INFO  [WorkerReceiver[myid=5]:FastLeaderElection@600] - Notification: 1 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,045 [myid:] - WARN  [SendWorker:1:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715)
2016-07-21 08:11:46,045 [myid:] - WARN  [SendWorker:1:QuorumCnxManager$SendWorker@736] - Send worker leaving thread
2016-07-21 08:11:46,045 [myid:] - INFO  [Thread-2:NIOServerCnxn@1008] - Closed socket connection for client /127.0.0.1:52363 (no session established for client)
2016-07-21 08:11:46,044 [myid:] - INFO  [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,044 [myid:] - INFO  [WorkerReceiver[myid=2]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,044 [myid:] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 4 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,046 [myid:] - INFO  [WorkerReceiver[myid=3]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,045 [myid:] - INFO  [/127.0.0.1:12227:QuorumCnxManager$Listener@541] - Received connection request /127.0.0.1:43446
2016-07-21 08:11:46,045 [myid:] - INFO  [WorkerReceiver[myid=5]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,045 [myid:] - WARN  [RecvWorker:5:QuorumCnxManager$RecvWorker@810] - Connection broken for id 5, my id = 1, error = 
java.io.EOFException
	at java.io.DataInputStream.readInt(DataInputStream.java:392)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795)
2016-07-21 08:11:46,047 [myid:] - WARN  [RecvWorker:5:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker
2016-07-21 08:11:46,045 [myid:] - WARN  [RecvWorker:1:QuorumCnxManager$RecvWorker@810] - Connection broken for id 1, my id = 5, error = 
java.net.SocketException: Socket closed
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at java.net.SocketInputStream.read(SocketInputStream.java:210)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795)
2016-07-21 08:11:46,047 [myid:] - WARN  [RecvWorker:1:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker
2016-07-21 08:11:46,047 [myid:] - INFO  [WorkerReceiver[myid=5]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 4 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,047 [myid:] - WARN  [SendWorker:5:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715)
2016-07-21 08:11:46,048 [myid:] - WARN  [SendWorker:5:QuorumCnxManager$SendWorker@736] - Send worker leaving thread
2016-07-21 08:11:46,046 [myid:] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,048 [myid:] - INFO  [WorkerReceiver[myid=5]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,048 [myid:] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 5 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,049 [myid:] - INFO  [WorkerReceiver[myid=5]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,049 [myid:] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 5 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,049 [myid:] - INFO  [WorkerReceiver[myid=5]:FastLeaderElection@600] - Notification: 1 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEpoch) LOOKING (my state)
2016-07-21 08:11:46,245 [myid:] - INFO  [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:QuorumPeer@844] - FOLLOWING
2016-07-21 08:11:46,246 [myid:] - INFO  [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11223:QuorumPeer@844] - FOLLOWING
2016-07-21 08:11:46,247 [myid:] - INFO  [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:11224:QuorumPeer@844] - FOLLOWING
2016-07-21 08:11:46,249 [myid:] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11222:QuorumPeer@844] - FOLLOWING
2016-07-21 08:11:46,249 [myid:] - INFO  [QuorumPeer[myid=5]/0:0:0:0:0:0:0:0:11226:QuorumPeer@856] - LEADING
2016-07-21 08:11:46,251 [myid:] - INFO  [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:Learner@86] - TCP NoDelay set to: true
2016-07-21 08:11:46,255 [myid:] - INFO  [QuorumPeer[myid=5]/0:0:0:0:0:0:0:0:11226:Leader@59] - TCP NoDelay set to: true
2016-07-21 08:11:46,259 [myid:] - INFO  [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:Environment@100] - Server environment:zookeeper.version=3.4.9-SNAPSHOT-1753645, built on 07/21/2016 07:46 GMT
2016-07-21 08:11:46,259 [myid:] - INFO  [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:Environment@100] - Server environment:host.name=asf907.gq1.ygridcore.net
2016-07-21 08:11:46,260 [myid:] - INFO  [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:Environment@100] - Server environment:java.version=1.7.0_80
2016-07-21 08:11:46,260 [myid:] - INFO  [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:Environment@100] - Server environment:java.vendor=Oracle Corporation
2016-07-21 08:11:46,260 [myid:] - INFO  [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:Environment@100] - Server environment:java.home=/home/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.7/jre
2016-07-21 08:11:46,260 [myid:] - INFO  [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:Environment@100] - Server environment:java.class.path=/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/test/classes:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/test/lib/antlr-2.7.6.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/test/lib/checkstyle-5.0.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/test/lib/commons-beanutils-core-1.7.0.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/test/lib/commons-cli-1.0.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/test/lib/commons-collections-3.2.2.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/test/lib/commons-lang-1.0.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/test/lib/commons-logging-1.0.3.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/test/lib/google-collections-0.9.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/test/lib/junit-4.8.1.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/test/lib/mockito-all-1.8.2.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/classes:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/src/java/lib/ivy-2.4.0.jar:/home/jenkins/tools/ant/latest/lib/ant.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/lib/jline-0.9.94.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/lib/log4j-1.2.16.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/lib/netty-3.10.5.Final.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/lib/slf4j-api-1.6.1.jar:/home/jenkins/jenkins-slave/workspace/ZooKeeper_branch34_jdk7/branch-3.4/build/lib/slf4j-log4j12-1.6.1.jar:/home/jenkins/tools/ant/apache-ant-1.9.4/lib/ant-launcher.jar:/home/jenkins/tools/ant/latest/lib/ant-junit.jar:/home/jenkins/tools/ant/latest/lib/ant-junit4.jar
2016-07-21 08:11:46,260 [myid:] - INFO  [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:Environment@100] - Server environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2016-07-21 08:11:46,260 [myid:] - INFO  [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:Environment@100] - Server environment:java.io.tmpdir=/tmp
2016-07-21 08:11:46,260 [myid:] - INFO  [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:Environment@100] - Server environment:java.compiler=<NA>
2016-07-21 08:11:46,260 [myid:] - INFO  [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:Environment@100] - Server environment:os.name=Linux
2016-07-21 08:11:46,260 [myid:] - INFO  [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:Environment@100] - Server environment:os.arch=amd64
2016-07-21 08:11:46,260 [myid:] - INFO  [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11225:Environment@100] - Server environment:os.version=3.13.0-36-lowlatency
2016-07-21 08:11:46,260 [myid:] - INFO  [QuorumPeer[myid=4]/0:0:0:
...[truncated 1420555 chars]...
ager$SendWorker.run(QuorumCnxManager.java:715)
2016-07-21 08:14:21,678 [myid:] - WARN  [SendWorker:2:QuorumCnxManager$SendWorker@736] - Send worker leaving thread
2016-07-21 08:14:21,672 [myid:] - WARN  [RecvWorker:1:QuorumCnxManager$RecvWorker@810] - Connection broken for id 1, my id = 5, error = 
java.io.EOFException
	at java.io.DataInputStream.readInt(DataInputStream.java:392)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795)
2016-07-21 08:14:21,679 [myid:] - WARN  [RecvWorker:1:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker
2016-07-21 08:14:21,672 [myid:] - WARN  [SendWorker:5:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715)
2016-07-21 08:14:21,679 [myid:] - WARN  [SendWorker:5:QuorumCnxManager$SendWorker@736] - Send worker leaving thread
2016-07-21 08:14:21,679 [myid:] - WARN  [SendWorker:1:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715)
2016-07-21 08:14:21,679 [myid:] - WARN  [SendWorker:1:QuorumCnxManager$SendWorker@736] - Send worker leaving thread
2016-07-21 08:14:21,678 [myid:] - WARN  [SendWorker:1:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715)
2016-07-21 08:14:21,679 [myid:] - WARN  [SendWorker:1:QuorumCnxManager$SendWorker@736] - Send worker leaving thread
2016-07-21 08:14:21,677 [myid:] - WARN  [SendWorker:1:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715)
2016-07-21 08:14:21,680 [myid:] - WARN  [SendWorker:1:QuorumCnxManager$SendWorker@736] - Send worker leaving thread
2016-07-21 08:14:21,675 [myid:] - WARN  [SendWorker:1:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715)
2016-07-21 08:14:21,680 [myid:] - WARN  [SendWorker:1:QuorumCnxManager$SendWorker@736] - Send worker leaving thread
2016-07-21 08:14:22,000 [myid:] - INFO  [SessionTracker:SessionTrackerImpl@162] - SessionTrackerImpl exited loop!
2016-07-21 08:14:22,317 [myid:] - INFO  [/127.0.0.1:12241:QuorumCnxManager$Listener@560] - Leaving listener
2016-07-21 08:14:22,320 [myid:] - INFO  [/127.0.0.1:12242:QuorumCnxManager$Listener@560] - Leaving listener
2016-07-21 08:14:22,579 [myid:] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11233:Follower@166] - shutdown called
java.lang.Exception: shutdown Follower
	at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:166)
	at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:850)
2016-07-21 08:14:22,579 [myid:] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11233:FollowerZooKeeperServer@140] - Shutting down
2016-07-21 08:14:22,579 [myid:] - INFO  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11233:SyncRequestProcessor@209] - Shutting down
2016-07-21 08:14:22,579 [myid:] - WARN  [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:11233:QuorumPeer@874] - QuorumPeer main thread exited
2016-07-21 08:14:22,582 [myid:] - INFO  [main:QuorumBase@306] - Shutting down quorum peer QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11234
2016-07-21 08:14:22,582 [myid:] - INFO  [main:Follower@166] - shutdown called
java.lang.Exception: shutdown Follower
	at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:166)
	at org.apache.zookeeper.server.quorum.QuorumPeer.shutdown(QuorumPeer.java:891)
	at org.apache.zookeeper.test.QuorumBase.shutdown(QuorumBase.java:307)
	at org.apache.zookeeper.test.QuorumBase.shutdownServers(QuorumBase.java:298)
	at org.apache.zookeeper.test.QuorumBase.tearDown(QuorumBase.java:285)
	at org.apache.zookeeper.test.QuorumZxidSyncTest.tearDown(QuorumZxidSyncTest.java:169)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:37)
	at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:48)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
	at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:532)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1179)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:1030)
2016-07-21 08:14:22,582 [myid:] - INFO  [main:FollowerZooKeeperServer@140] - Shutting down
2016-07-21 08:14:22,582 [myid:] - INFO  [main:ZooKeeperServer@469] - shutting down
2016-07-21 08:14:22,582 [myid:] - INFO  [main:FollowerRequestProcessor@107] - Shutting down
2016-07-21 08:14:22,582 [myid:] - INFO  [main:CommitProcessor@184] - Shutting down
2016-07-21 08:14:22,582 [myid:] - INFO  [FollowerRequestProcessor:2:FollowerRequestProcessor@97] - FollowerRequestProcessor exited loop!
2016-07-21 08:14:22,582 [myid:] - INFO  [CommitProcessor:2:CommitProcessor@153] - CommitProcessor exited loop!
2016-07-21 08:14:22,582 [myid:] - INFO  [main:FinalRequestProcessor@402] - shutdown of request processor complete
2016-07-21 08:14:22,583 [myid:] - INFO  [main:SyncRequestProcessor@209] - Shutting down
2016-07-21 08:14:22,583 [myid:] - INFO  [SyncThread:2:SyncRequestProcessor@187] - SyncRequestProcessor exited!
2016-07-21 08:14:22,584 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11234:NIOServerCnxnFactory@219] - NIOServerCnxn factory exited run method
2016-07-21 08:14:22,585 [myid:] - WARN  [SendWorker:3:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715)
2016-07-21 08:14:22,585 [myid:] - WARN  [RecvWorker:2:QuorumCnxManager$RecvWorker@810] - Connection broken for id 2, my id = 4, error = 
java.io.EOFException
	at java.io.DataInputStream.readInt(DataInputStream.java:392)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795)
2016-07-21 08:14:22,585 [myid:] - WARN  [RecvWorker:2:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker
2016-07-21 08:14:22,585 [myid:] - WARN  [RecvWorker:3:QuorumCnxManager$RecvWorker@810] - Connection broken for id 3, my id = 2, error = 
java.net.SocketException: Socket closed
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at java.net.SocketInputStream.read(SocketInputStream.java:210)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795)
2016-07-21 08:14:22,586 [myid:] - WARN  [RecvWorker:3:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker
2016-07-21 08:14:22,585 [myid:] - WARN  [RecvWorker:2:QuorumCnxManager$RecvWorker@810] - Connection broken for id 2, my id = 5, error = 
java.io.EOFException
	at java.io.DataInputStream.readInt(DataInputStream.java:392)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795)
2016-07-21 08:14:22,586 [myid:] - WARN  [RecvWorker:2:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker
2016-07-21 08:14:22,585 [myid:] - WARN  [RecvWorker:5:QuorumCnxManager$RecvWorker@810] - Connection broken for id 5, my id = 2, error = 
java.net.SocketException: Socket closed
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at java.net.SocketInputStream.read(SocketInputStream.java:210)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795)
2016-07-21 08:14:22,587 [myid:] - WARN  [RecvWorker:5:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker
2016-07-21 08:14:22,585 [myid:] - WARN  [SendWorker:3:QuorumCnxManager$SendWorker@736] - Send worker leaving thread
2016-07-21 08:14:22,585 [myid:] - WARN  [RecvWorker:2:QuorumCnxManager$RecvWorker@810] - Connection broken for id 2, my id = 3, error = 
java.io.EOFException
	at java.io.DataInputStream.readInt(DataInputStream.java:392)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795)
2016-07-21 08:14:22,587 [myid:] - WARN  [RecvWorker:2:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker
2016-07-21 08:14:22,585 [myid:] - WARN  [SendWorker:5:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715)
2016-07-21 08:14:22,587 [myid:] - WARN  [SendWorker:5:QuorumCnxManager$SendWorker@736] - Send worker leaving thread
2016-07-21 08:14:22,585 [myid:] - ERROR [/127.0.0.1:12239:QuorumCnxManager$Listener@547] - Exception while listening
java.net.SocketException: Socket closed
	at java.net.PlainSocketImpl.socketAccept(Native Method)
	at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
	at java.net.ServerSocket.implAccept(ServerSocket.java:530)
	at java.net.ServerSocket.accept(ServerSocket.java:498)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCnxManager.java:539)
2016-07-21 08:14:22,587 [myid:] - WARN  [SendWorker:2:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715)
2016-07-21 08:14:22,588 [myid:] - WARN  [SendWorker:2:QuorumCnxManager$SendWorker@736] - Send worker leaving thread
2016-07-21 08:14:22,587 [myid:] - WARN  [SendWorker:2:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715)
2016-07-21 08:14:22,589 [myid:] - WARN  [SendWorker:2:QuorumCnxManager$SendWorker@736] - Send worker leaving thread
2016-07-21 08:14:22,586 [myid:] - WARN  [SendWorker:4:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715)
2016-07-21 08:14:22,589 [myid:] - WARN  [SendWorker:4:QuorumCnxManager$SendWorker@736] - Send worker leaving thread
2016-07-21 08:14:22,586 [myid:] - WARN  [RecvWorker:4:QuorumCnxManager$RecvWorker@810] - Connection broken for id 4, my id = 2, error = 
java.net.SocketException: Socket closed
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at java.net.SocketInputStream.read(SocketInputStream.java:210)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795)
2016-07-21 08:14:22,589 [myid:] - WARN  [RecvWorker:4:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker
2016-07-21 08:14:22,586 [myid:] - INFO  [main:QuorumBase@310] - Shutting down leader election QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11234
2016-07-21 08:14:22,589 [myid:] - INFO  [main:QuorumBase@315] - Waiting for QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11234 to exit thread
2016-07-21 08:14:22,585 [myid:] - WARN  [SendWorker:2:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715)
2016-07-21 08:14:22,590 [myid:] - WARN  [SendWorker:2:QuorumCnxManager$SendWorker@736] - Send worker leaving thread
2016-07-21 08:14:22,673 [myid:] - INFO  [/127.0.0.1:12238:QuorumCnxManager$Listener@560] - Leaving listener
2016-07-21 08:14:23,079 [myid:] - INFO  [WorkerSender[myid=3]:FastLeaderElection$Messenger$WorkerSender@438] - WorkerSender is down
2016-07-21 08:14:23,080 [myid:] - INFO  [WorkerSender[myid=4]:FastLeaderElection$Messenger$WorkerSender@438] - WorkerSender is down
2016-07-21 08:14:23,084 [myid:] - INFO  [WorkerSender[myid=5]:FastLeaderElection$Messenger$WorkerSender@438] - WorkerSender is down
2016-07-21 08:14:23,088 [myid:] - INFO  [WorkerReceiver[myid=3]:FastLeaderElection$Messenger$WorkerReceiver@407] - WorkerReceiver is down
2016-07-21 08:14:23,088 [myid:] - INFO  [WorkerReceiver[myid=4]:FastLeaderElection$Messenger$WorkerReceiver@407] - WorkerReceiver is down
2016-07-21 08:14:23,089 [myid:] - INFO  [WorkerReceiver[myid=5]:FastLeaderElection$Messenger$WorkerReceiver@407] - WorkerReceiver is down
2016-07-21 08:14:23,578 [myid:] - INFO  [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11234:Follower@166] - shutdown called
java.lang.Exception: shutdown Follower
	at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:166)
	at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:850)
2016-07-21 08:14:23,578 [myid:] - INFO  [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11234:FollowerZooKeeperServer@140] - Shutting down
2016-07-21 08:14:23,578 [myid:] - INFO  [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11234:SyncRequestProcessor@209] - Shutting down
2016-07-21 08:14:23,578 [myid:] - WARN  [QuorumPeer[myid=2]/0:0:0:0:0:0:0:0:11234:QuorumPeer@874] - QuorumPeer main thread exited
2016-07-21 08:14:23,579 [myid:] - INFO  [main:QuorumBase@306] - Shutting down quorum peer QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:11235
2016-07-21 08:14:23,579 [myid:] - INFO  [main:Follower@166] - shutdown called
java.lang.Exception: shutdown Follower
	at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:166)
	at org.apache.zookeeper.server.quorum.QuorumPeer.shutdown(QuorumPeer.java:891)
	at org.apache.zookeeper.test.QuorumBase.shutdown(QuorumBase.java:307)
	at org.apache.zookeeper.test.QuorumBase.shutdownServers(QuorumBase.java:299)
	at org.apache.zookeeper.test.QuorumBase.tearDown(QuorumBase.java:285)
	at org.apache.zookeeper.test.QuorumZxidSyncTest.tearDown(QuorumZxidSyncTest.java:169)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:37)
	at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:48)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
	at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:532)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1179)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:1030)
2016-07-21 08:14:23,579 [myid:] - INFO  [main:FollowerZooKeeperServer@140] - Shutting down
2016-07-21 08:14:23,579 [myid:] - INFO  [main:ZooKeeperServer@469] - shutting down
2016-07-21 08:14:23,580 [myid:] - INFO  [main:FollowerRequestProcessor@107] - Shutting down
2016-07-21 08:14:23,580 [myid:] - INFO  [main:CommitProcessor@184] - Shutting down
2016-07-21 08:14:23,580 [myid:] - INFO  [FollowerRequestProcessor:3:FollowerRequestProcessor@97] - FollowerRequestProcessor exited loop!
2016-07-21 08:14:23,580 [myid:] - INFO  [CommitProcessor:3:CommitProcessor@153] - CommitProcessor exited loop!
2016-07-21 08:14:23,580 [myid:] - INFO  [main:FinalRequestProcessor@402] - shutdown of request processor complete
2016-07-21 08:14:23,581 [myid:] - INFO  [main:SyncRequestProcessor@209] - Shutting down
2016-07-21 08:14:23,581 [myid:] - INFO  [SyncThread:3:SyncRequestProcessor@187] - SyncRequestProcessor exited!
2016-07-21 08:14:23,582 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11235:NIOServerCnxnFactory@219] - NIOServerCnxn factory exited run method
2016-07-21 08:14:23,583 [myid:] - ERROR [/127.0.0.1:12240:QuorumCnxManager$Listener@547] - Exception while listening
java.net.SocketException: Socket closed
	at java.net.PlainSocketImpl.socketAccept(Native Method)
	at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
	at java.net.ServerSocket.implAccept(ServerSocket.java:530)
	at java.net.ServerSocket.accept(ServerSocket.java:498)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCnxManager.java:539)
2016-07-21 08:14:23,583 [myid:] - WARN  [SendWorker:5:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715)
2016-07-21 08:14:23,585 [myid:] - WARN  [SendWorker:5:QuorumCnxManager$SendWorker@736] - Send worker leaving thread
2016-07-21 08:14:23,584 [myid:] - WARN  [RecvWorker:4:QuorumCnxManager$RecvWorker@810] - Connection broken for id 4, my id = 3, error = 
java.net.SocketException: Socket closed
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at java.net.SocketInputStream.read(SocketInputStream.java:210)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795)
2016-07-21 08:14:23,585 [myid:] - WARN  [RecvWorker:4:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker
2016-07-21 08:14:23,584 [myid:] - WARN  [SendWorker:4:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715)
2016-07-21 08:14:23,585 [myid:] - WARN  [SendWorker:4:QuorumCnxManager$SendWorker@736] - Send worker leaving thread
2016-07-21 08:14:23,584 [myid:] - WARN  [RecvWorker:5:QuorumCnxManager$RecvWorker@810] - Connection broken for id 5, my id = 3, error = 
java.net.SocketException: Socket closed
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at java.net.SocketInputStream.read(SocketInputStream.java:210)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795)
2016-07-21 08:14:23,585 [myid:] - WARN  [RecvWorker:5:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker
2016-07-21 08:14:23,584 [myid:] - WARN  [RecvWorker:3:QuorumCnxManager$RecvWorker@810] - Connection broken for id 3, my id = 5, error = 
java.io.EOFException
	at java.io.DataInputStream.readInt(DataInputStream.java:392)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795)
2016-07-21 08:14:23,586 [myid:] - WARN  [RecvWorker:3:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker
2016-07-21 08:14:23,584 [myid:] - WARN  [RecvWorker:3:QuorumCnxManager$RecvWorker@810] - Connection broken for id 3, my id = 4, error = 
java.io.EOFException
	at java.io.DataInputStream.readInt(DataInputStream.java:392)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795)
2016-07-21 08:14:23,586 [myid:] - WARN  [RecvWorker:3:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker
2016-07-21 08:14:23,584 [myid:] - INFO  [main:QuorumBase@310] - Shutting down leader election QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:11235
2016-07-21 08:14:23,586 [myid:] - INFO  [main:QuorumBase@315] - Waiting for QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:11235 to exit thread
2016-07-21 08:14:23,586 [myid:] - WARN  [SendWorker:3:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715)
2016-07-21 08:14:23,586 [myid:] - WARN  [SendWorker:3:QuorumCnxManager$SendWorker@736] - Send worker leaving thread
2016-07-21 08:14:23,586 [myid:] - WARN  [SendWorker:3:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715)
2016-07-21 08:14:23,587 [myid:] - WARN  [SendWorker:3:QuorumCnxManager$SendWorker@736] - Send worker leaving thread
2016-07-21 08:14:23,588 [myid:] - INFO  [/127.0.0.1:12239:QuorumCnxManager$Listener@560] - Leaving listener
2016-07-21 08:14:24,355 [myid:] - INFO  [WorkerSender[myid=2]:FastLeaderElection$Messenger$WorkerSender@438] - WorkerSender is down
2016-07-21 08:14:24,358 [myid:] - INFO  [WorkerSender[myid=1]:FastLeaderElection$Messenger$WorkerSender@438] - WorkerSender is down
2016-07-21 08:14:24,360 [myid:] - INFO  [WorkerSender[myid=3]:FastLeaderElection$Messenger$WorkerSender@438] - WorkerSender is down
2016-07-21 08:14:24,361 [myid:] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection$Messenger$WorkerReceiver@407] - WorkerReceiver is down
2016-07-21 08:14:24,361 [myid:] - INFO  [WorkerReceiver[myid=2]:FastLeaderElection$Messenger$WorkerReceiver@407] - WorkerReceiver is down
2016-07-21 08:14:24,362 [myid:] - INFO  [WorkerReceiver[myid=3]:FastLeaderElection$Messenger$WorkerReceiver@407] - WorkerReceiver is down
2016-07-21 08:14:24,578 [myid:] - INFO  [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:11235:Follower@166] - shutdown called
java.lang.Exception: shutdown Follower
	at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:166)
	at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:850)
2016-07-21 08:14:24,578 [myid:] - INFO  [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:11235:FollowerZooKeeperServer@140] - Shutting down
2016-07-21 08:14:24,578 [myid:] - INFO  [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:11235:SyncRequestProcessor@209] - Shutting down
2016-07-21 08:14:24,578 [myid:] - WARN  [QuorumPeer[myid=3]/0:0:0:0:0:0:0:0:11235:QuorumPeer@874] - QuorumPeer main thread exited
2016-07-21 08:14:24,579 [myid:] - INFO  [main:QuorumBase@306] - Shutting down quorum peer QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11236
2016-07-21 08:14:24,579 [myid:] - INFO  [main:Follower@166] - shutdown called
java.lang.Exception: shutdown Follower
	at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:166)
	at org.apache.zookeeper.server.quorum.QuorumPeer.shutdown(QuorumPeer.java:891)
	at org.apache.zookeeper.test.QuorumBase.shutdown(QuorumBase.java:307)
	at org.apache.zookeeper.test.QuorumBase.shutdownServers(QuorumBase.java:300)
	at org.apache.zookeeper.test.QuorumBase.tearDown(QuorumBase.java:285)
	at org.apache.zookeeper.test.QuorumZxidSyncTest.tearDown(QuorumZxidSyncTest.java:169)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:37)
	at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:48)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
	at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:532)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1179)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:1030)
2016-07-21 08:14:24,580 [myid:] - INFO  [main:FollowerZooKeeperServer@140] - Shutting down
2016-07-21 08:14:24,581 [myid:] - INFO  [main:ZooKeeperServer@469] - shutting down
2016-07-21 08:14:24,581 [myid:] - INFO  [main:FollowerRequestProcessor@107] - Shutting down
2016-07-21 08:14:24,581 [myid:] - INFO  [main:CommitProcessor@184] - Shutting down
2016-07-21 08:14:24,581 [myid:] - INFO  [main:FinalRequestProcessor@402] - shutdown of request processor complete
2016-07-21 08:14:24,581 [myid:] - INFO  [FollowerRequestProcessor:4:FollowerRequestProcessor@97] - FollowerRequestProcessor exited loop!
2016-07-21 08:14:24,581 [myid:] - INFO  [main:SyncRequestProcessor@209] - Shutting down
2016-07-21 08:14:24,581 [myid:] - INFO  [CommitProcessor:4:CommitProcessor@153] - CommitProcessor exited loop!
2016-07-21 08:14:24,582 [myid:] - INFO  [SyncThread:4:SyncRequestProcessor@187] - SyncRequestProcessor exited!
2016-07-21 08:14:24,583 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11236:NIOServerCnxnFactory@219] - NIOServerCnxn factory exited run method
2016-07-21 08:14:24,585 [myid:] - INFO  [main:QuorumBase@310] - Shutting down leader election QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11236
2016-07-21 08:14:24,586 [myid:] - ERROR [/127.0.0.1:12241:QuorumCnxManager$Listener@547] - Exception while listening
java.net.SocketException: Socket closed
	at java.net.PlainSocketImpl.socketAccept(Native Method)
	at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
	at java.net.ServerSocket.implAccept(ServerSocket.java:530)
	at java.net.ServerSocket.accept(ServerSocket.java:498)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCnxManager.java:539)
2016-07-21 08:14:24,586 [myid:] - WARN  [SendWorker:5:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715)
2016-07-21 08:14:24,586 [myid:] - WARN  [SendWorker:5:QuorumCnxManager$SendWorker@736] - Send worker leaving thread
2016-07-21 08:14:24,586 [myid:] - INFO  [main:QuorumBase@315] - Waiting for QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11236 to exit thread
2016-07-21 08:14:24,586 [myid:] - WARN  [RecvWorker:4:QuorumCnxManager$RecvWorker@810] - Connection broken for id 4, my id = 5, error = 
java.io.EOFException
	at java.io.DataInputStream.readInt(DataInputStream.java:392)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795)
2016-07-21 08:14:24,587 [myid:] - WARN  [RecvWorker:4:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker
2016-07-21 08:14:24,585 [myid:] - INFO  [/127.0.0.1:12240:QuorumCnxManager$Listener@560] - Leaving listener
2016-07-21 08:14:24,585 [myid:] - WARN  [RecvWorker:5:QuorumCnxManager$RecvWorker@810] - Connection broken for id 5, my id = 4, error = 
java.net.SocketException: Socket closed
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at java.net.SocketInputStream.read(SocketInputStream.java:210)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:795)
2016-07-21 08:14:24,588 [myid:] - WARN  [RecvWorker:5:QuorumCnxManager$RecvWorker@813] - Interrupting SendWorker
2016-07-21 08:14:24,588 [myid:] - WARN  [SendWorker:4:QuorumCnxManager$SendWorker@727] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:879)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:65)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:715)
2016-07-21 08:14:24,588 [myid:] - WARN  [SendWorker:4:QuorumCnxManager$SendWorker@736] - Send worker leaving thread
2016-07-21 08:14:25,579 [myid:] - INFO  [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11236:Follower@166] - shutdown called
java.lang.Exception: shutdown Follower
	at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:166)
	at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:850)
2016-07-21 08:14:25,579 [myid:] - INFO  [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11236:FollowerZooKeeperServer@140] - Shutting down
2016-07-21 08:14:25,579 [myid:] - INFO  [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11236:SyncRequestProcessor@209] - Shutting down
2016-07-21 08:14:25,579 [myid:] - WARN  [QuorumPeer[myid=4]/0:0:0:0:0:0:0:0:11236:QuorumPeer@874] - QuorumPeer main thread exited
2016-07-21 08:14:25,579 [myid:] - INFO  [main:QuorumBase@306] - Shutting down quorum peer QuorumPeer[myid=5]/0:0:0:0:0:0:0:0:11237
2016-07-21 08:14:25,579 [myid:] - INFO  [main:Leader@496] - Shutting down
2016-07-21 08:14:25,580 [myid:] - INFO  [main:Leader@502] - Shutdown called
java.lang.Exception: shutdown Leader! reason: quorum Peer shutdown
	at org.apache.zookeeper.server.quorum.Leader.shutdown(Leader.java:502)
	at org.apache.zookeeper.server.quorum.QuorumPeer.shutdown(QuorumPeer.java:888)
	at org.apache.zookeeper.test.QuorumBase.shutdown(QuorumBase.java:307)
	at org.apache.zookeeper.test.QuorumBase.shutdownServers(QuorumBase.java:301)
	at org.apache.zookeeper.test.QuorumBase.tearDown(QuorumBase.java:285)
	at org.apache.zookeeper.test.QuorumZxidSyncTest.tearDown(QuorumZxidSyncTest.java:169)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:37)
	at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:48)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
	at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:39)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:532)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1179)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:1030)
2016-07-21 08:14:25,581 [myid:] - INFO  [main:ZooKeeperServer@469] - shutting down
2016-07-21 08:14:25,581 [myid:] - INFO  [LearnerCnxAcceptor-/127.0.0.1:12237:Leader$LearnerCnxAcceptor@325] - exception while shutting down acceptor: java.net.SocketException: Socket closed
2016-07-21 08:14:25,582 [myid:] - INFO  [main:SessionTrackerImpl@225] - Shutting down
2016-07-21 08:14:25,582 [myid:] - INFO  [main:PrepRequestProcessor@765] - Shutting down
2016-07-21 08:14:25,582 [myid:] - INFO  [main:ProposalRequestProcessor@88] - Shutting down
2016-07-21 08:14:25,582 [myid:] - INFO  [main:CommitProcessor@184] - Shutting down
2016-07-21 08:14:25,582 [myid:] - INFO  [ProcessThread(sid:5 cport:-1)::PrepRequestProcessor@143] - PrepRequestProcessor exited loop!
2016-07-21 08:14:25,583 [myid:] - INFO  [CommitProcessor:5:CommitProcessor@153] - CommitProcessor exited loop!
2016-07-21 08:14:25,582 [myid:] - INFO  [main:Leader$ToBeAppliedRequestProcessor@661] - Shutting down
2016-07-21 08:14:25,583 [myid:] - INFO  [main:FinalRequestProcessor@402] - shutdown of request processor complete
2016-07-21 08:14:25,583 [myid:] - INFO  [main:SyncRequestProcessor@209] - Shutting down
2016-07-21 08:14:25,583 [myid:] - INFO  [SyncThread:5:SyncRequestProcessor@187] - SyncRequestProcessor exited!
2016-07-21 08:14:25,585 [myid:] - WARN  [LearnerHandler-/127.0.0.1:43485:LearnerHandler@644] - ******* GOODBYE /127.0.0.1:43485 ********
2016-07-21 08:14:25,585 [myid:] - WARN  [LearnerHandler-/127.0.0.1:43486:LearnerHandler@644] - ******* GOODBYE /127.0.0.1:43486 ********
2016-07-21 08:14:25,585 [myid:] - WARN  [LearnerHandler-/127.0.0.1:43486:LearnerHandler@656] - Ignoring unexpected exception
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1219)
	at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:340)
	at java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:338)
	at org.apache.zookeeper.server.quorum.LearnerHandler.shutdown(LearnerHandler.java:654)
	at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:647)
2016-07-21 08:14:25,585 [myid:] - WARN  [LearnerHandler-/127.0.0.1:43485:LearnerHandler@656] - Ignoring unexpected exception
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1219)
	at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:340)
	at java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:338)
	at org.apache.zookeeper.server.quorum.LearnerHandler.shutdown(LearnerHandler.java:654)
	at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:647)
2016-07-21 08:14:25,585 [myid:] - WARN  [LearnerHandler-/127.0.0.1:43488:LearnerHandler@644] - ******* GOODBYE /127.0.0.1:43488 ********
2016-07-21 08:14:25,586 [myid:] - INFO  [/127.0.0.1:12241:QuorumCnxManager$Listener@560] - Leaving listener
2016-07-21 08:14:25,586 [myid:] - WARN  [LearnerHandler-/127.0.0.1:43487:LearnerHandler@644] - ******* GOODBYE /127.0.0.1:43487 ********
2016-07-21 08:14:25,586 [myid:] - WARN  [LearnerHandler-/127.0.0.1:43487:LearnerHandler@656] - Ignoring unexpected exception
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1219)
	at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:340)
	at java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:338)
	at org.apache.zookeeper.server.quorum.LearnerHandler.shutdown(LearnerHandler.java:654)
	at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:647)
2016-07-21 08:14:25,586 [myid:] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:11237:NIOServerCnxnFactory@219] - NIOServerCnxn factory exited run method
2016-07-21 08:14:25,586 [myid:] - WARN  [LearnerHandler-/127.0.0.1:43488:LearnerHandler@656] - Ignoring unexpected exception
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1219)
	at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:340)
	at java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:338)
	at org.apache.zookeeper.server.quorum.LearnerHandler.shutdown(LearnerHandler.java:654)
	at org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:647)
2016-07-21 08:14:25,587 [myid:] - WARN  [QuorumPeer[myid=5]/0:0:0:0:0:0:0:0:11237:QuorumPeer@862] - Unexpected exception
java.lang.InterruptedException: sleep interrupted
	at java.lang.Thread.sleep(Native Method)
	at org.apache.zookeeper.server.quorum.Leader.lead(Leader.java:456)
	at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:859)
2016-07-21 08:14:25,588 [myid:] - INFO  [QuorumPeer[myid=5]/0:0:0:0:0:0:0:0:11237:Leader@496] - Shutting down
2016-07-21 08:14:25,588 [myid:] - INFO  [main:QuorumBase@310] - Shutting down leader election QuorumPeer[myid=5]/0:0:0:0:0:0:0:0:11237
2016-07-21 08:14:25,588 [myid:] - ERROR [/127.0.0.1:12242:QuorumCnxManager$Listener@547] - Exception while listening
java.net.SocketException: Socket closed
	at java.net.PlainSocketImpl.socketAccept(Native Method)
	at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
	at java.net.ServerSocket.implAccept(ServerSocket.java:530)
	at java.net.ServerSocket.accept(ServerSocket.java:498)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCnxManager.java:539)
2016-07-21 08:14:25,588 [myid:] - INFO  [main:QuorumBase@315] - Waiting for QuorumPeer[myid=5]/0:0:0:0:0:0:0:0:11237 to exit thread
2016-07-21 08:14:25,588 [myid:] - WARN  [QuorumPeer[myid=5]/0:0:0:0:0:0:0:0:11237:QuorumPeer@874] - QuorumPeer main thread exited
2016-07-21 08:14:25,589 [myid:] - INFO  [main:FourLetterWordMain@62] - connecting to 127.0.0.1 11233
2016-07-21 08:14:25,589 [myid:] - INFO  [main:QuorumBase@291] - 127.0.0.1:11233 is no longer accepting client connections
2016-07-21 08:14:25,589 [myid:] - INFO  [main:FourLetterWordMain@62] - connecting to 127.0.0.1 11234
2016-07-21 08:14:25,589 [myid:] - INFO  [main:QuorumBase@291] - 127.0.0.1:11234 is no longer accepting client connections
2016-07-21 08:14:25,590 [myid:] - INFO  [main:FourLetterWordMain@62] - connecting to 127.0.0.1 11235
2016-07-21 08:14:25,590 [myid:] - INFO  [main:QuorumBase@291] - 127.0.0.1:11235 is no longer accepting client connections
2016-07-21 08:14:25,590 [myid:] - INFO  [main:FourLetterWordMain@62] - connecting to 127.0.0.1 11236
2016-07-21 08:14:25,590 [myid:] - INFO  [main:QuorumBase@291] - 127.0.0.1:11236 is no longer accepting client connections
2016-07-21 08:14:25,590 [myid:] - INFO  [main:FourLetterWordMain@62] - connecting to 127.0.0.1 11237
2016-07-21 08:14:25,590 [myid:] - INFO  [main:QuorumBase@291] - 127.0.0.1:11237 is no longer accepting client connections
2016-07-21 08:14:25,592 [myid:] - INFO  [main:ZKTestCase$1@60] - SUCCEEDED testLateLogs
2016-07-21 08:14:25,592 [myid:] - INFO  [main:ZKTestCase$1@55] - FINISHED testLateLogs

Attachments

Issue Links

relates to

ZOOKEEPER-2135 fix trunk build

Open

Activity

People

Assignee:: Andor Molnar

Reporter:: Michael Han

Votes:: 0 Vote for this issue

Watchers:: 2 Start watching this issue

Dates

Created:: 21/Jul/16 22:44

Updated:: 25/Oct/18 14:55

Resolved:: 25/Oct/18 14:55