Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-24471

The way we bootstrap meta table is confusing

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.0.0-alpha-1, 2.4.0
    • master, meta, proc-v2
    • None
    • Incompatible change, Reviewed
    • Hide
      Move all the meta initialization code in MasterFileSystem and HRegionServer to InitMetaProcedure. Add a new step for InitMetaProcedure called INIT_META_WRITE_FS_LAYOUT to place the moved code.

      This is an incompatible change, but should not have much impact. InitMetaProcedure will only be executed once when bootstraping a fresh new cluster, so typically this will not effect rolling upgrading. And even if you hit this problem, as long as InitMetaProcedure has not been finished, we can make sure that there is no user data in the cluster, you can just clean up the cluster and try again. There will be no data loss.
      Show
      Move all the meta initialization code in MasterFileSystem and HRegionServer to InitMetaProcedure. Add a new step for InitMetaProcedure called INIT_META_WRITE_FS_LAYOUT to place the moved code. This is an incompatible change, but should not have much impact. InitMetaProcedure will only be executed once when bootstraping a fresh new cluster, so typically this will not effect rolling upgrading. And even if you hit this problem, as long as InitMetaProcedure has not been finished, we can make sure that there is no user data in the cluster, you can just clean up the cluster and try again. There will be no data loss.

    Description

      I used to thought the work is only done in the InitMetaProcedure but in fact we have several pieces of code related to this which are placed in different classes.

      In HRegionServer.initializeFileSystem, we will test whether we are HMaster. If so, we will call FSTableDescriptors.tryUpdateMetaTableDescriptor. And in MasterFileSystem.checkRootDir, we will check if meta region is available. If not, we will bootstrap it, and it will also call FSTableDescriptors.tryUpdateMetaTableDescriptor.

      The problem here is that, HRegionServer.initializeFileSystem is called in the constructor of HRegionServer, which means it will be executed before MasterFileSystem.checkRootDir, as we will only create the MasterFileSystem for active master...

      Ideally I prefer we move all the bootstrap code into InitMetaProcedure, which will be much cleaner. If this can not be done, at least we could move the code into MasterFileSystem.

      Attachments

        Issue Links

          causes

          Sub-task - The sub-task of the issue HBASE-24833 Bootstrap should not delete the META table directory if it's not partial

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #1806

          Activity

            People

              zhangduo Duo Zhang
              zhangduo Duo Zhang
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: