Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-9878

Rolling Updates 2.0 to 2.1 "unable to gossip"

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Normal
    • Resolution: Duplicate
    • None
    • None
    • None
    • Normal

    Description

      Hi there,

      we are currently testing an upgrade of our servers from Cassandra 2.0.16 to 2.1.8 on the EC2 service of Amazon

      Usually, we launch a new server which gets the newest version and then joins the existing ring, bootstrapping, and after some time we kill one of the old nodes.

      But with the upgrade to 2.1 the new server will not join the existing ring.

      INFO  [HANDSHAKE-/10.] 2015-07-23 11:06:14,997 OutboundTcpConnection.java:485 - Handshaking version with /10.xx.yy.zz
      INFO  [HANDSHAKE-/10.] 2015-07-23 11:06:14,999 OutboundTcpConnection.java:485 - Handshaking version with /10.yy.zz.aa
      INFO  [HANDSHAKE-/10.] 2015-07-23 11:06:15,000 OutboundTcpConnection.java:485 - Handshaking version with /10.aa.bb.cc
      ERROR [main] 2015-07-23 11:06:46,016 CassandraDaemon.java:541 - Exception encountered during startup
      java.lang.RuntimeException: Unable to gossip with any seeds
              at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1307) ~[apache-cassandra-2.1.8.jar:2.1.8]
              at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:533) ~[apache-cassandra-2.1.8.jar:2.1.8]
              at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:777) ~[apache-cassandra-2.1.8.jar:2.1.8]
              at org.apache.cassandra.service.StorageService.initServer(StorageService.java:714) ~[apache-cassandra-2.1.8.jar:2.1.8]
              at org.apache.cassandra.service.StorageService.initServer(StorageService.java:605) ~[apache-cassandra-2.1.8.jar:2.1.8]
              at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:378) [apache-cassandra-2.1.8.jar:2.1.8]
              at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:524) [apache-cassandra-2.1.8.jar:2.1.8]
              at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:613) [apache-cassandra-2.1.8.jar:2.1.8]
      WARN  [StorageServiceShutdownHook] 2015-07-23 11:06:46,023 Gossiper.java:1418 - No local state or state is in silent shutdown, not announcing shutdown
      INFO  [StorageServiceShutdownHook] 2015-07-23 11:06:46,023 MessagingService.java:708 - Waiting for messaging service to quiesce
      INFO  [ACCEPT-/10.] 2015-07-23 11:06:46,045 MessagingService.java:958 - MessagingService has terminated the accept() thread
      
      

      our config uses the "RandomPartitioner", "SimpleSnitch" and the internal nodes ip for communication.

      when i use the same config but ONLY 2.1.x Servers, everthing works perfectly, but as soon as we start to "mix in" a new version into an "old" ring, the new servers will not "gossip"... ( so it cannot be a firewall issue, or such)

      If you need any more information from my side, please let me know.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              akaempfe@megazebra.com Axel Kämpfe
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: