Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-23634 Enable "Split WAL to HFile" by default
  3. HBASE-24734

RegionInfo#containsRange should support check meta table

    XMLWordPrintableJSON

Details

    Description

      Came across this when we were testing the 'split-to-hfile' feature running ITBLL:

       

      2020-07-10 10:16:49,983 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closing region hbase:meta,,1.15882307402020-07-10 10:16:49,997 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed hbase:meta,,1.15882307402020-07-10 10:16:49,998 WARN org.apache.hadoop.hbase.regionserver.handler.AssignRegionHandler: Fatal error occurred while opening region hbase:meta,,1.1588230740, aborting...java.lang.IllegalArgumentException: Invalid range: IntegrationTestBigLinkedList,,1594350463222.8f89e01a5245e79946e22d8a8ab4698b. > IntegrationTestBigLinkedList,\x10\x02J\xA1,1594349535271.be24dc276f686e6dcc7fb9d3f91c8387.        at org.apache.hadoop.hbase.client.RegionInfoBuilder$MutableRegionInfo.containsRange(RegionInfoBuilder.java:300)        at org.apache.hadoop.hbase.regionserver.HStore.tryCommitRecoveredHFile(HStore.java:1111)        at org.apache.hadoop.hbase.regionserver.HRegion.loadRecoveredHFilesIfAny(HRegion.java:5442)        at org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:1010)        at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:950)        at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7490)        at org.apache.hadoop.hbase.regionserver.HRegion.openHRegionFromTableDir(HRegion.java:7448)        at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7424)        at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7382)        at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:7333)        at org.apache.hadoop.hbase.regionserver.handler.AssignRegionHandler.process(AssignRegionHandler.java:135)        at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:104)        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)        at java.base/java.lang.Thread.run(Thread.java:834)2020-07-10 10:16:50,005 ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: ***** ABORTING region server hbasedn149.example.org,16020,1594375563853: Failed to open region hbase:meta,,1.1588230740 and can not recover *****java.lang.IllegalArgumentException: Invalid range: IntegrationTestBigLinkedList,,1594350463222.8f89e01a5245e79946e22d8a8ab4698b. > IntegrationTestBigLinkedList,\x10\x02J\xA1,1594349535271.be24dc276f686e6dcc7fb9d3f91c8387.
      
       

      Seems basic case of wrong comparator. Below passes if I use the meta comparator

       @Test
      public void testBinaryKeys() throws Exception {
        Set<Cell> set = new TreeSet<>(CellComparatorImpl.COMPARATOR);
        final byte [] fam = Bytes.toBytes("col");
        final byte [] qf = Bytes.toBytes("umn");
        final byte [] nb = new byte[0];
        Cell [] keys = {
            createByteBufferKeyValueFromKeyValue(
                new KeyValue(Bytes.toBytes("aaaaa,\u0000\u0000,2"), fam, qf, 2, nb)),
            createByteBufferKeyValueFromKeyValue(
                new KeyValue(Bytes.toBytes("aaaaa,\u0001,3"), fam, qf, 3, nb)),
            createByteBufferKeyValueFromKeyValue(
                new KeyValue(Bytes.toBytes("aaaaa,,1"), fam, qf, 1, nb)),
            createByteBufferKeyValueFromKeyValue(
                new KeyValue(Bytes.toBytes("aaaaa,\u1000,5"), fam, qf, 5, nb)),
            createByteBufferKeyValueFromKeyValue(
                new KeyValue(Bytes.toBytes("aaaaa,a,4"), fam, qf, 4, nb)),
            createByteBufferKeyValueFromKeyValue(
                new KeyValue(Bytes.toBytes("a,a,0"), fam, qf, 0, nb)),
        };
        // Add to set with bad comparator
        Collections.addAll(set, keys);
        // This will output the keys incorrectly.
        boolean assertion = false;
        int count = 0;
        try {
          for (Cell k: set) {
            assertTrue("count=" + count + ", " + k.toString(), count++ == k.getTimestamp());
          }
        } catch (AssertionError e) {
          // Expected
          assertion = true;
        }
        assertTrue(assertion);
        // Make set with good comparator
        set = new TreeSet<>(CellComparatorImpl.META_COMPARATOR);
        Collections.addAll(set, keys);
        count = 0;
        for (Cell k: set) {
          assertTrue("count=" + count + ", " + k.toString(), count++ == k.getTimestamp());
        }
      }

      Attachments

        Activity

          People

            meiyi Yi Mei
            stack Michael Stack
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: