Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-25389

[Flakey Tests] branch-2 TestMetaShutdownHandler

    XMLWordPrintableJSON

Details

    • Task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.0.0-alpha-1, 2.5.0, 2.4.1
    • flakies
    • None
    • Reviewed

    Description

      I see this in local runs fail regularly. We kill the server hosting meta and then check it came up in a new location after waiting on recovery. In the test, when it fails, the assert on new location fails because we have not waited for the CRASH to happen. Here is excerpt from log:

       2020-12-11 13:20:27,298 INFO  [Listener at localhost/62149] master.TestMetaShutdownHandler(111): Deleted the znode for the RegionServer hosting hbase:meta; waiting on SSH
      ...
       2020-12-11 13:20:27,310 INFO  [Listener at localhost/62149] master.TestMetaShutdownHandler(122): Past wait on RIT
      ...
      
       2020-12-11 13:20:27,351 DEBUG [RegionServerTracker-0] procedure2.ProcedureExecutor(1048): Stored pid=9, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure stack.XXX.example.com,62201,1607721618377, splitWal=true, meta=true
      

      The first line is where we remove the ephemeral node for the regionserver carrying hbase:meta. The second line is supposed to log AFTER SCP is done (it calls it SSH in this old test above). Notice how the 3rd line, after the 2nd, is first mention of SCP being queued.

      Attachments

        Issue Links

          Activity

            People

              stack Michael Stack
              stack Michael Stack
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: