Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-20006

TestRestoreSnapshotFromClientWithRegionReplicas is flakey

    XMLWordPrintableJSON

Details

    Description

      Failing 10% of the time. Interestingly, it is below that causes fail. We go to split but it is already split. We will then fail the split with an internal assert which messes up procedures; at a minimum we should just not split (this is in the prepare stage).

      2018-02-15 23:21:42,162 INFO  [PEWorker-12] procedure.MasterProcedureScheduler(571): pid=105, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testOnlineSnapshotAfterSplittingRegions-1518736887838, parent=3f850cea7d71a7ebd019f2f009efca4d, daughterA=06b5e6366efbef155d70e56cfdf58dc9, daughterB=8c175de1b33765a5683ac1e502edb0bd, table=testOnlineSnapshotAfterSplittingRegions-1518736887838, testOnlineSnapshotAfterSplittingRegions-1518736887838,,1518736887882.3f850cea7d71a7ebd019f2f009efca4d.
      2018-02-15 23:21:42,162 INFO  [PEWorker-12] assignment.SplitTableRegionProcedure(440): Split of {ENCODED => 3f850cea7d71a7ebd019f2f009efca4d, NAME => 'testOnlineSnapshotAfterSplittingRegions-1518736887838,,1518736887882.3f850cea7d71a7ebd019f2f009efca4d.', STARTKEY => '', ENDKEY => '1'} skipped; state is already SPLIT
      2018-02-15 23:21:42,163 ERROR [PEWorker-12] procedure2.ProcedureExecutor(1480): CODE-BUG: Uncaught runtime exception: pid=105, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testOnlineSnapshotAfterSplittingRegions-1518736887838, parent=3f850cea7d71a7ebd019f2f009efca4d, daughterA=06b5e6366efbef155d70e56cfdf58dc9, daughterB=8c175de1b33765a5683ac1e502edb0bd
      java.lang.AssertionError: split region should have an exception here
        at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:228)
        at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:89)
        at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:180)
        at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:845)
        at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1455)
        at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1224)
        at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$800(ProcedureExecutor.java:78)
        at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1734)
      

      Attachments

        1. HBASE-20006.branch-2.001.patch
          3 kB
          Michael Stack
        2. HBASE-20006.master.001.patch
          5 kB
          Toshihiro Suzuki
        3. HBASE-20006.master.002.patch
          7 kB
          Toshihiro Suzuki
        4. HBASE-20006.master.003.patch
          8 kB
          Toshihiro Suzuki
        5. HBASE-20006.master.003.patch
          8 kB
          Toshihiro Suzuki

        Issue Links

          Activity

            People

              brfrn169 Toshihiro Suzuki
              stack Michael Stack
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: