Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-10233

IndexOutOfBoundsException in HintedHandOffManager

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Low
    • Resolution: Fixed
    • 2.1.12, 2.2.4
    • None
    • None
    • Cassandra 2.2.0

    • Low

    Description

      After upgrading our cluster to 2.2.0, the following error started showing exectly every 10 minutes on every server in the cluster:

      INFO  [CompactionExecutor:1381] 2015-08-31 18:31:55,506 CompactionTask.java:142 - Compacting (8e7e1520-500e-11e5-b1e3-e95897ba4d20) [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-540-big-Data.db:level=0, ]
      INFO  [CompactionExecutor:1381] 2015-08-31 18:31:55,599 CompactionTask.java:224 - Compacted (8e7e1520-500e-11e5-b1e3-e95897ba4d20) 1 sstables to [/cassandra/data/system/hints-2666e20573ef38b390fefecf96e8f0c7/la-541-big,] to level=0.  1,544,495 bytes to 1,544,495 (~100% of original) in 93ms = 15.838121MB/s.  0 total partitions merged to 4.  Partition merge counts were {1:4, }
      ERROR [HintedHandoff:1] 2015-08-31 18:31:55,600 CassandraDaemon.java:182 - Exception in thread Thread[HintedHandoff:1,1,main]
      java.lang.IndexOutOfBoundsException: null
      	at java.nio.Buffer.checkIndex(Buffer.java:538) ~[na:1.7.0_79]
      	at java.nio.HeapByteBuffer.getLong(HeapByteBuffer.java:410) ~[na:1.7.0_79]
      	at org.apache.cassandra.utils.UUIDGen.getUUID(UUIDGen.java:106) ~[apache-cassandra-2.2.0.jar:2.2.0]
      	at org.apache.cassandra.db.HintedHandOffManager.scheduleAllDeliveries(HintedHandOffManager.java:515) ~[apache-cassandra-2.2.0.jar:2.2.0]
      	at org.apache.cassandra.db.HintedHandOffManager.access$000(HintedHandOffManager.java:88) ~[apache-cassandra-2.2.0.jar:2.2.0]
      	at org.apache.cassandra.db.HintedHandOffManager$1.run(HintedHandOffManager.java:168) ~[apache-cassandra-2.2.0.jar:2.2.0]
      	at org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118) ~[apache-cassandra-2.2.0.jar:2.2.0]
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_79]
      	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) [na:1.7.0_79]
      	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) [na:1.7.0_79]
      	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [na:1.7.0_79]
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_79]
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_79]
      	at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79]
      

      Attachments

        1. cassandra-2.2.1-10233-v2.txt
          1 kB
          J.P. Eiti Kimura
        2. cassandra-2.0-10233.txt
          1 kB
          J.P. Eiti Kimura
        3. cassandra-2.1-10233-v5.txt
          1 kB
          J.P. Eiti Kimura
        4. cassandra-2.1.8-10233-v4.txt
          3 kB
          J.P. Eiti Kimura
        5. cassandra-2.2.1-10233.txt
          2 kB
          J.P. Eiti Kimura
        6. cassandra-2.1.8-10233-v3.txt
          5 kB
          J.P. Eiti Kimura

        Issue Links

          Activity

            People

              eitikimura J.P. Eiti Kimura
              mroi Omri Iluz
              J.P. Eiti Kimura
              Paulo Motta
              Votes:
              13 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: