Details
-
Bug
-
Status: Resolved
-
Normal
-
Resolution: Not A Bug
-
We have a Cassandra Cluster with 2 Datacenters:
Cassandra Version: 2.2.5
DC1 - OperationalDC:
3 nodes Virtual Server on VMWare.
S.O.: Windows 2012 Server 64bits-native
vCPU 12 Cores @ 2.5Ghz
RAM: 16GB
HDD: 3,6 TB Virtual DriveDC2 - AnalyticalDC:
3 nodes Virtual Server on VMWare.
S.O.: Running on Windows 2012 Server 64bits-native
vCPU 12 Cores @ 2.5Ghz
RAM: 16GB
HDD: 3,6 TB Virtual DriveWe have a Cassandra Cluster with 2 Datacenters: Cassandra Version: 2.2.5 DC1 - OperationalDC: 3 nodes Virtual Server on VMWare. S.O.: Windows 2012 Server 64bits-native vCPU 12 Cores @ 2.5Ghz RAM: 16GB HDD: 3,6 TB Virtual Drive DC2 - AnalyticalDC: 3 nodes Virtual Server on VMWare. S.O.: Running on Windows 2012 Server 64bits-native vCPU 12 Cores @ 2.5Ghz RAM: 16GB HDD: 3,6 TB Virtual Drive
-
Normal
Description
Our application connects directly to OperationalDC via CQL.
During the daily operation we have a large number of WriteTimeoutException and ReadTimeoutException errors.
In the logs of Cassandra we can see that the read and write timeout errors are also recorded with the error:
Java.lang.RuntimeException: java.lang.RuntimeException: org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 1 responses.
Log Detail, Application Level:
_22:54:22,945 [ERROR] AbstractEntityManager - Insert object 'Event
' error com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency LOCAL_QUORUM (2 replica were required but only 1 acknowledged the write)_
Log Detail, Cassandra Level:
EBUG [SharedPool-Worker-6] 2017-08-24 08:36:40,114 StorageProxy.java:1893 - Range slice failure; received 0 of 1 responses for range 1 of 1 java.lang.RuntimeException: java.lang.RuntimeException: org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out - received only 1 responses.
The timeout error occurs 2 seconds later because of the configuration in the yaml:underlined text
_# How long the coordinator should wait for writes to complete
write_request_timeout_in_ms: 2000_
