Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-10353

NULL in compaction_history in 2.2.1

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Normal
    • Resolution: Duplicate
    • None
    • None
    • None
    • Normal

    Description

      While doing some tests on 2.2.1 I found a problem with a null value in the compaction_history table that results in a nullpointer exception when doing 'nodetool compactionhistory'.

      $ ccm node1 nodetool compactionhistory
      Compaction History:
      error: null
      – StackTrace –
      java.lang.NullPointerException
      at com.google.common.base.Joiner$MapJoiner.join(Joiner.java:330)
      at org.apache.cassandra.utils.FBUtilities.toString(FBUtilities.java:477)
      at org.apache.cassandra.db.compaction.CompactionHistoryTabularData.from(CompactionHistoryTabularData.java:78)
      at org.apache.cassandra.db.SystemKeyspace.getCompactionHistory(SystemKeyspace.java:425)
      at org.apache.cassandra.db.compaction.CompactionManager.getCompactionHistory(CompactionManager.java:1492)

      The problem is the null value in the rows_merged column.

      cqlsh> SELECT * FROM system.compaction_history ;

      id | bytes_in | bytes_out | columnfamily_name | compacted_at | keyspace_name | rows_merged
      -------------------------------------------------------------------------------------------------------------------------------------
      86548100-5adf-11e5-a4f4-29d398af617a | 366 | 64 | compactions_in_progress | 2015-09-14 12:52:58+0000 | system | {2: 2}
      9ff24610-5adf-11e5-a4f4-29d398af617a | 30 | 0 | bar | 2015-09-14 12:53:41+0000 | foo | null
      e4ffb130-5ade-11e5-a4f4-29d398af617a | 366 | 64 | compactions_in_progress | 2015-09-14 12:48:27+0000 | system | {2: 2}

      After some more testing and digging in the code I realised that this is the result of a compaction where all sstables are dropped (i.e. no new sstable is created).

      I tried to recreate this using 2.1.9 and found that there where nothing inserted into the compaction_history table if the same thing happens. That is due to a check in CompactionTask.runMayThrow() line 181.

      if (!iter.hasNext())
      {
          // don't mark compacted in the finally block, since if there _is_ nondeleted data,
          // we need to sync it (via closeAndOpen) first, so there is no period during which
          // a crash could cause data loss.
          cfs.markObsolete(sstables, compactionType);
          return;
      }
      

      The result of the return here is that we end the compaction without inserting anything in the compaction_history table, 2.2.1 does not have this logic and will insert the null value.

      Not sure what the best solution would be:
      1. Implement the same behaviour as in 2.1.9 and skip inserting anything into compaction_history.
      2. Make sure we insert a more sensible value then null, but I'm not sure what that value would be.
      3. Have nodetool handle the null value without a nullpointer exception.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              tommy_s Tommy Stendahl
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: