Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-10423

Paxos/LWT failures when moving node

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Normal
    • Resolution: Duplicate
    • None
    • Cassandra version: 2.0.14
      Java-driver version: 2.0.11

    • Normal

    Description

      While moving a node (nodetool move <newtoken>) we noticed that lwt started failing for some (~50%) requests. The java-driver (version 2.0.11) returned com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency SERIAL (7 replica were required but only 0 acknowledged the write). The cluster was not under heavy load.

      I noticed that the failed lwt requests all took just above 1s. That information and the WriteTimeoutException could indicate that this happens:
      https://github.com/apache/cassandra/blob/cassandra-2.0.14/src/java/org/apache/cassandra/service/StorageProxy.java#L268
      I can't explain why though. Why would there be more cas contention just because a node is moving?

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              rschildmeijer Roger Schildmeijer
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: