Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-27091

Speed up the loading of table descriptor from filesystem

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 3.0.0-alpha-3
    • 3.0.0-alpha-4
    • master
    • None
    • Reviewed
    • Hide
      Loading table descriptors with multiple threads is supported after HBASE-27091 (but turned off by default). The thread number could be configured via `hbase.tabledescriptor.parallel.load.threads`, and setting it back to zero will turn off the feature.
      Show
      Loading table descriptors with multiple threads is supported after HBASE-27091 (but turned off by default). The thread number could be configured via `hbase.tabledescriptor.parallel.load.threads`, and setting it back to zero will turn off the feature.

    Description

      If there are a large number of tables in the HBase cluster, it will take a long time to fully load the table descriptor  from filesystem for the first time.
      In our production cluster, there were 50000 + tables. It took several minutes to load the table descriptor from filesystem. This problem seriously affects the performance of HMaster active/standby switchover.

      We should support concurrent loading to solve this problem.

      Attachments

        Issue Links

          relates to

          Task - A task that needs to be done. HBASE-27839 Backport HBASE-27091 to branch-2

          • Major - Major loss of function.
          • Open
          links to

          Web Link GitHub Pull Request #4493

          Activity

            People

              heliangjun Liangjun He
              heliangjun Liangjun He
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: