Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-25832

Performance improvements for StochasticLoadBalancer

    XMLWordPrintableJSON

Details

    • Umbrella
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • Balancer, Performance
    • None

    Description

      This is what I posted in the slack channel.

      When refactoring balancer related code for moving them out to hbase-balancer module, I found that we will create a OffPeakHours instance in MoveCostFunction.cost every time. So I wrote a simple JMH test to test its performance.

      https://github.com/Apache9/hbase-jmh-test/blob/master/src/main/java/org/apache/hadoop/hbase/master/balancer/MoveCostFunctionBenchmark.java

      For the original version, the result is
      Benchmark Mode Samples Score Score error Units
      o.a.h.h.m.b.MoveCostFunctionBenchmark.computeCost avgt 10 242.312 3.853 ns/op
      12:06
      If I initialized OffPeakHours in constructor, then the result is
      Benchmark Mode Samples Score Score error Units
      o.a.h.h.m.b.MoveCostFunctionBenchmark.computeCost avgt 10 81.186 0.720 ns/op
      12:06
      If I also eliminate the conf.getFloat in the cost method(by storing them to a field member in constructor, the result is
      Benchmark Mode Samples Score Score error Units
      o.a.h.h.m.b.MoveCostFunctionBenchmark.computeCost avgt 10 4.419 0.035 ns/op
      12:08
      I think on a large cluster, the performance for balancer is a problem, so maybe we should try to avoid creating too many objects or calling conf.getXXX in the critical path? As we will call StochasticLoadBalancer.computeCost in every step when balancing cluster...

      For large clusters, the performance of StochasticLoadBalancer will be very important, so I think we need to treat this problem seriously and try to improve.

      Attachments

        Issue Links

          Activity

            People

              zhangduo Duo Zhang
              zhangduo Duo Zhang
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated: