Uploaded image for project: 'Geode'
  1. Geode
  2. GEODE-8267

serverRestartsAfterOneLocatorDies hangs

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None

    Description

      hang: https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/DistributedTestOpenJDK11/builds/275#A

       

      The test hung in serverRestartsAfterOneLocatorDies after another failure in the same test class.

      Here's the hung thread:

      "Test worker" #27 prio=5 os_prio=0 cpu=5016.73ms elapsed=5638.52s tid=0x00007f01c8ad4800 nid=0x18 runnable  [0x00007f019872c000]"Test worker" #27 prio=5 os_prio=0 cpu=5016.73ms elapsed=5638.52s tid=0x00007f01c8ad4800 nid=0x18 runnable  [0x00007f019872c000]   java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(java.base@11.0.7/Native Method) at java.net.SocketInputStream.socketRead(java.base@11.0.7/SocketInputStream.java:115) at java.net.SocketInputStream.read(java.base@11.0.7/SocketInputStream.java:168) at java.net.SocketInputStream.read(java.base@11.0.7/SocketInputStream.java:140) at java.io.BufferedInputStream.fill(java.base@11.0.7/BufferedInputStream.java:252) at java.io.BufferedInputStream.read(java.base@11.0.7/BufferedInputStream.java:271) - locked <0x00000000d08fe7a0> (a java.io.BufferedInputStream) at java.io.DataInputStream.readByte(java.base@11.0.7/DataInputStream.java:270) at sun.rmi.transport.StreamRemoteCall.executeCall(java.rmi@11.0.7/StreamRemoteCall.java:240) at sun.rmi.server.UnicastRef.invoke(java.rmi@11.0.7/UnicastRef.java:164) at java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(java.rmi@11.0.7/RemoteObjectInvocationHandler.java:217) at java.rmi.server.RemoteObjectInvocationHandler.invoke(java.rmi@11.0.7/RemoteObjectInvocationHandler.java:162) at com.sun.proxy.$Proxy53.executeMethodOnObject(Unknown Source) at org.apache.geode.test.dunit.VM.executeMethodOnObject(VM.java:607) at org.apache.geode.test.dunit.VM.invoke(VM.java:450) at org.apache.geode.test.dunit.rules.ClusterStartupRule.startServerVM(ClusterStartupRule.java:268) at org.apache.geode.test.dunit.rules.ClusterStartupRule.startServerVM(ClusterStartupRule.java:261) at org.apache.geode.test.dunit.rules.ClusterStartupRule.startServerVM(ClusterStartupRule.java:256) at org.apache.geode.management.internal.configuration.ClusterConfigLocatorRestartDUnitTest.serverRestartsAfterOneLocatorDies(ClusterConfigLocatorRestartDUnitTest.java:114) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@11.0.7/Native Method) 

      Here's the previous test failure, which may have affected the test that hung:

      org.apache.geode.management.internal.configuration.ClusterConfigLocatorRestartDUnitTest > serverRestartHangsWaitingForStartupMessageResponse FAILED
      
          org.junit.runners.model.TestTimedOutException: test timed out after 300000 milliseconds
      
              at java.net.SocketInputStream.socketRead0(Native Method)
      
              at java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
      
              at java.net.SocketInputStream.read(SocketInputStream.java:168)
      
              at java.net.SocketInputStream.read(SocketInputStream.java:140)
      
              at java.io.BufferedInputStream.fill(BufferedInputStream.java:252)
      
              at java.io.BufferedInputStream.read(BufferedInputStream.java:271)
      
              at java.io.DataInputStream.readByte(DataInputStream.java:270)
      
              at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:240)
      
              at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:164)
      
              at java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(RemoteObjectInvocationHandler.java:217)
      
              at java.rmi.server.RemoteObjectInvocationHandler.invoke(RemoteObjectInvocationHandler.java:162)
      
              at com.sun.proxy.$Proxy53.executeMethodOnObject(Unknown Source)
      
              at org.apache.geode.test.dunit.VM.executeMethodOnObject(VM.java:607)
      
              at org.apache.geode.test.dunit.VM.invoke(VM.java:437)
      
              at org.apache.geode.test.junit.rules.VMProvider.invoke(VMProvider.java:94)
      
              at org.apache.geode.management.internal.configuration.ClusterConfigLocatorRestartDUnitTest.serverRestartHangsWaitingForStartupMessageResponse(ClusterConfigLocatorRestartDUnitTest.java:176)
      

      Seems like 300s should be long enough so I fear there may be a real problem here.

      Attachments

        Activity

          People

            Unassigned Unassigned
            burcham Bill Burcham
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: