Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-6759

If Load-Balanced Connections encounter IOException, it will loop indefinitely, logging errors instead of closing the connection and moving on

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • 1.10.0
    • 1.10.0
    • Core Framework
    • None

    Description

      I encountered an Exception when communicating between two nodes using load-balanced connections. Within 2 minutes, I had geneated 2.5 GB of error logs, all of the pattern:

      2019-10-09 14:36:35,091 ERROR [Load-Balance Server Thread-17] o.a.n.c.q.c.s.ConnectionLoadBalanceServer Failed to communicate with Peer localhost/127.0.0.1:61671
      java.net.SocketException: Socket is closed
         at sun.security.ssl.SSLSocketImpl.checkEOF(SSLSocketImpl.java:1532)
         at sun.security.ssl.AppInputStream.read(AppInputStream.java:95)
         at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
         at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
         at org.apache.nifi.controller.queue.clustered.server.StandardLoadBalanceProtocol.negotiateProtocolVersion(StandardLoadBalanceProtocol.java:143)
         at org.apache.nifi.controller.queue.clustered.server.StandardLoadBalanceProtocol.receiveFlowFiles(StandardLoadBalanceProtocol.java:122)
         at org.apache.nifi.controller.queue.clustered.server.ConnectionLoadBalanceServer$CommunicateAction.run(ConnectionLoadBalanceServer.java:152)
         at java.lang.Thread.run(Thread.java:745)
      2019-10-09 14:36:35,091 ERROR [Load-Balance Server Thread-5] o.a.n.c.q.c.s.ConnectionLoadBalanceServer Failed to communicate with Peer localhost/127.0.0.1:61644
      java.net.SocketException: Socket is closed
         at sun.security.ssl.SSLSocketImpl.checkEOF(SSLSocketImpl.java:1532)
         at sun.security.ssl.AppInputStream.read(AppInputStream.java:95)
         at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
         at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
         at org.apache.nifi.controller.queue.clustered.server.StandardLoadBalanceProtocol.negotiateProtocolVersion(StandardLoadBalanceProtocol.java:143)
         at org.apache.nifi.controller.queue.clustered.server.StandardLoadBalanceProtocol.receiveFlowFiles(StandardLoadBalanceProtocol.java:122)
         at org.apache.nifi.controller.queue.clustered.server.ConnectionLoadBalanceServer$CommunicateAction.run(ConnectionLoadBalanceServer.java:152)
         at java.lang.Thread.run(Thread.java:745)
       

      Attachments

        Issue Links

          Activity

            People

              markap14 Mark Payne
              markap14 Mark Payne
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h
                  1h