Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-13103

[ergonomics] add region size balancing as a feature of master

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 1.2.0, 1.3.0, 2.0.0
    • Balancer, Usability
    • None
    • Reviewed
    • Hide
      This patch adds optional ability for HMaster to normalize regions in size (disabled by default, change hbase.normalizer.enabled property to true to turn it on). If enabled, HMaster periodically (every 30 minutes by default) monitors tables for which normalization is enabled in table configuration and performs splits/merges as seems appropriate. Users may implement their own normalization strategies by implementing RegionNormalizer interface and configuring it in hbase-site.xml.
      Show
      This patch adds optional ability for HMaster to normalize regions in size (disabled by default, change hbase.normalizer.enabled property to true to turn it on). If enabled, HMaster periodically (every 30 minutes by default) monitors tables for which normalization is enabled in table configuration and performs splits/merges as seems appropriate. Users may implement their own normalization strategies by implementing RegionNormalizer interface and configuring it in hbase-site.xml.

    Description

      Often enough, folks miss-judge split points or otherwise end up with a suboptimal number of regions. We should have an automated, reliable way to "reshape" or "balance" a table's region boundaries. This would be for tables that contain existing data. This might look like:

      Admin#reshapeTable(TableName, int numSplits);

      or from the shell:

      > reshape TABLE, numSplits

      Better still would be to have a maintenance process, similar to the existing Balancer that runs AssignmentManager on an interval, to run the above "reshape" operation on an interval. That way, the cluster will automatically self-correct toward a desirable state.

      Attachments

        1. HBASE-13103-branch-1.v3.patch
          57 kB
          Mikhail Antonov
        2. HBASE-13103-v0.patch
          49 kB
          Mikhail Antonov
        3. HBASE-13103-v1.patch
          53 kB
          Mikhail Antonov
        4. HBASE-13103-v2.patch
          56 kB
          Mikhail Antonov
        5. HBASE-13103-v3.patch
          56 kB
          Mikhail Antonov

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. HBASE-14838 Clarify that SimpleRegionNormalizer does not merge empty (<1MB) regions

          • Trivial - Cosmetic problem like misspelt words or misaligned text.
          • Closed

          Task - A task that needs to be done. HBASE-13964 Skip region normalization for tables under namespace quota

          • Major - Major loss of function.
          • Closed

          Task - A task that needs to be done. HBASE-15073 Finer grained control over normalization actions for RegionNormalizer

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HBASE-15098 Normalizer switch in configuration is not used

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HBASE-14367 Add normalization support to shell

          • Major - Major loss of function.
          • Closed

          Task - A task that needs to be done. HBASE-15124 Document the new 'normalization' feature in refguid

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          supercedes

          Bug - A problem which impairs or prevents the functions of the product. HBASE-7629 Master initiated automatic region merges

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-3521 region be merged with others automatically when all data in the region has expired and removed, or region gets too small.

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          links to

          Web Link review board

          Activity

            People

              mantonov Mikhail Antonov
              ndimiduk Nick Dimiduk
              Votes:
              0 Vote for this issue
              Watchers:
              32 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: