Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-19383

Restarting a node causes MV unavailable exceptions afterward

    XMLWordPrintableJSON

Details

    • Availability - Unavailable
    • Normal
    • Normal
    • User Report
    • All
    • None

    Description

      If a cluster is started with materialized views enabled and a client is constantly writing to one, when a node is restarted shortly after startup it may throw UEs:

      INFO [GossipStage:1] 2024-02-09 06:17:10,073 Gossiper.java:1419 - Node /10.5.11.102:7000 has restarted, now UP
      ERROR [MutationStage-2] 2024-02-09 06:17:10,074 Keyspace.java:650 - Unknown exception caught while attempting to update MaterializedView! stresstest.transport_orders
      org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level ONE
      at org.apache.cassandra.exceptions.UnavailableException.create(UnavailableException.java:37)
      at org.apache.cassandra.locator.ReplicaPlans.assureSufficientLiveReplicas(ReplicaPlans.java:169)
      at org.apache.cassandra.locator.ReplicaPlans.assureSufficientLiveReplicasForWrite(ReplicaPlans.java:112)
      at org.apache.cassandra.locator.ReplicaPlans.forWrite(ReplicaPlans.java:353)
      at org.apache.cassandra.locator.ReplicaPlans.forWrite(ReplicaPlans.java:344)
      at org.apache.cassandra.locator.ReplicaPlans.forWrite(ReplicaPlans.java:338)
      at org.apache.cassandra.service.StorageProxy.wrapViewBatchResponseHandler(StorageProxy.java:1417)
      at org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:1077)
      at org.apache.cassandra.db.view.TableViews.pushViewReplicaUpdates(TableViews.java:169)
      at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:645)
      at org.apache.cassandra.db.Keyspace.applyFuture(Keyspace.java:476)
      at org.apache.cassandra.db.Mutation.applyFuture(Mutation.java:223)
      at org.apache.cassandra.db.MutationVerbHandler.doVerb(MutationVerbHandler.java:54)
      at org.apache.cassandra.net.InboundSink.lambda$new$0(InboundSink.java:78)
      at org.apache.cassandra.net.InboundSink.accept(InboundSink.java:97)
      at org.apache.cassandra.net.InboundSink.accept(InboundSink.java:45)
      at org.apache.cassandra.net.InboundMessageHandler$ProcessMessage.run(InboundMessageHandler.java:430)
      at org.apache.cassandra.concurrent.ExecutionFailure$1.run(ExecutionFailure.java:133)
      at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:142)
      at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
      at java.lang.Thread.run(Thread.java:750)

      It's not immediately clear what the impact of this is since the error is on restart and not communicated back to the client.

      Attachments

        Activity

          People

            Unassigned Unassigned
            brandon.williams Brandon Williams
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: