Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-24255

GCRegionProcedure doesn't assign region from RegionServer leading to orphans

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.2.4
    • None
    • None
    • hbase 2.2.4

      hadoop 3.1.3

    Description

      We've found ourselves in a situation where parents of merged or split regions needed to be opened again on a regionserver due to having to recover from cluster meltdown (HBCK2's fixMeta kicks off GCMultipleMergedRegionsProcedure which requiters all regions to be merged to be open). Then, when a GCProcedure is kicked of to clean a parent region up by GCMultipleMergedRegionsProcedure, it ends up deleting it from hbase:meta, but doesn't unassign it from RegionServer leading for it to show up in "Orphan Regions on RegionServer" in hbck tab of HBase Master. Also, the hbase client doesn't detect that the region is closed either because it's still technically open on a regionserver (it doesn't reread hbase:meta all the time). The only way to recover from this is to restart regionserver which isn't idea as it can lead to other issues in clusters with region inconsistencies.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              timoha Andrey Elenskiy
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated: