Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-17706

TableSkewCostFunction improperly computes max skew

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 1.2.0
    • 2.0.0
    • Balancer
    • CentOS Derivative with a derivative of the 3.18.43 kernel. HBase on CDH5.9.0 with some patches. HDFS CDH 5.9.0 with no patches.

    • Patch

    Description

      We noticed while running unit tests that the TableSkewCostFunction computed cost did not change as the balancer ran and simulated moves across the cluster. After investigating, we found that this happened in particular when the cluster started out with at least one table very strongly skewed.

      We noticed that the TableSkewCostFunction depends on a field of the BaseLoadBalancer.Cluster class called numMaxRegionsPerTable, but this field is not properly maintained as regionMoves are simulated for the cluster. The field only ever increases as the maximum number of regions per table increases, but it does not decrease as the maximum number per table goes down.

      This patch corrects that behavior so that the field is accurately maintained, and thus the TableSkewCostFunction produces a more correct value as the balancer runs.

      Attachments

        1. HBASE-17706.patch
          Delete this attachment
          2 kB
          Kahlil Oppenheimer
        2. HBASE-17706-01.patch
          Delete this attachment
          6 kB
          Kahlil Oppenheimer
        3. HBASE-17706-02.patch
          Delete this attachment
          6 kB
          Kahlil Oppenheimer
        4. HBASE-17706-03.patch
          Delete this attachment
          6 kB
          Kahlil Oppenheimer
        5. HBASE-17706-04.patch
          Delete this attachment
          6 kB
          Kahlil Oppenheimer
        6. HBASE-17706-05.patch
          Delete this attachment
          8 kB
          Kahlil Oppenheimer
        7. HBASE-17706-06.patch
          Delete this attachment
          6 kB
          Kahlil Oppenheimer
        8. HBASE-17706-07.patch
          Delete this attachment
          6 kB
          Kahlil Oppenheimer

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            kahliloppenheimer Kahlil Oppenheimer Assign to me
            kahliloppenheimer Kahlil Oppenheimer
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment