Details
-
Task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
-
Reviewed
Description
In ticket HBASE-17808, the author introduced a fastpath implementation for RWQueueRpcExecutor. It aggregated 3 different independent RpcExecutor to implement the mechanism. This redundancy costed more memory and from its own performance test, it cannot outperform the original implementation. This time, I directly extended RWQueueRpcExecutor to implement the fast path mechanism. From my test result, it has a better queue time performance than before.
YCSB Test:
Constant Configurations:
hbase.regionserver.handler.count: 1000
hbase.ipc.server.callqueue.read.ratio: 0.5
hbase.ipc.server.callqueue.handler.factor: 0.2
Test Workload:
YCSB: 50% Write, 25% Get, 25% Scan. Max Scan length: 1000.
Client Threads: 100
| FastPathRWQueueRpcExecutor | RWQueueRpcExecutor |
|---|---|
| [OVERALL], RunTime(ms), 909365 [OVERALL], Throughput(ops/sec), 5498.3422498116815 [TOTAL_GCS_PS_Scavenge], Count, 1208 [TOTAL_GC_TIME_PS_Scavenge], Time(ms), 8006 [TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 0.8803945610398465 [TOTAL_GCS_PS_MarkSweep], Count, 2 [TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 96 [TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.010556817119638429 [TOTAL_GCs], Count, 1210 [TOTAL_GC_TIME], Time(ms), 8102 [TOTAL_GC_TIME_%], Time(%), 0.8909513781594849 [READ], Operations, 1248885 [READ], AverageLatency(us), 14080.154160711354 [READ], MinLatency(us), 269 [READ], MaxLatency(us), 180735 [READ], 95thPercentileLatency(us), 29775 [READ], 99thPercentileLatency(us), 39391 [READ], Return=OK, 1248885 [CLEANUP], Operations, 200 [CLEANUP], AverageLatency(us), 311.78 [CLEANUP], MinLatency(us), 1 [CLEANUP], MaxLatency(us), 59647 [CLEANUP], 95thPercentileLatency(us), 26 [CLEANUP], 99thPercentileLatency(us), 173 [INSERT], Operations, 1251067 [INSERT], AverageLatency(us), 14235.898240461942 [INSERT], MinLatency(us), 393 [INSERT], MaxLatency(us), 204159 [INSERT], 95thPercentileLatency(us), 29919 [INSERT], 99thPercentileLatency(us), 39647 [INSERT], Return=OK, 1251067 [UPDATE], Operations, 1249582 [UPDATE], AverageLatency(us), 14166.923049467741 [UPDATE], MinLatency(us), 321 [UPDATE], MaxLatency(us), 203647 [UPDATE], 95thPercentileLatency(us), 29855 [UPDATE], 99thPercentileLatency(us), 39551 [UPDATE], Return=OK, 1249582 [SCAN], Operations, 1250466 [SCAN], AverageLatency(us), 30056.68854251135 [SCAN], MinLatency(us), 787 [SCAN], MaxLatency(us), 509183 [SCAN], 95thPercentileLatency(us), 57823 [SCAN], 99thPercentileLatency(us), 74751 [SCAN], Return=OK, 1250466 |
[OVERALL], RunTime(ms), 958763 [OVERALL], Throughput(ops/sec), 5215.053146606617 [TOTAL_GCS_PS_Scavenge], Count, 1264 [TOTAL_GC_TIME_PS_Scavenge], Time(ms), 8680 [TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 0.9053332262509086 [TOTAL_GCS_PS_MarkSweep], Count, 1 [TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 38 [TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.00396344039142103 [TOTAL_GCs], Count, 1265 [TOTAL_GC_TIME], Time(ms), 8718 [TOTAL_GC_TIME_%], Time(%), 0.9092966666423298 [READ], Operations, 1250961 [READ], AverageLatency(us), 14663.084518222391 [READ], MinLatency(us), 320 [READ], MaxLatency(us), 204415 [READ], 95thPercentileLatency(us), 30815 [READ], 99thPercentileLatency(us), 43071 [READ], Return=OK, 1250961 [CLEANUP], Operations, 200 [CLEANUP], AverageLatency(us), 366.845 [CLEANUP], MinLatency(us), 1 [CLEANUP], MaxLatency(us), 70719 [CLEANUP], 95thPercentileLatency(us), 36 [CLEANUP], 99thPercentileLatency(us), 80 [INSERT], Operations, 1248183 [INSERT], AverageLatency(us), 14334.938754974231 [INSERT], MinLatency(us), 390 [INSERT], MaxLatency(us), 2828287 [INSERT], 95thPercentileLatency(us), 30271 [INSERT], 99thPercentileLatency(us), 41919 [INSERT], Return=OK, 1248183 [UPDATE], Operations, 1250212 [UPDATE], AverageLatency(us), 14283.836318960304 [UPDATE], MinLatency(us), 337 [UPDATE], MaxLatency(us), 2828287 [UPDATE], 95thPercentileLatency(us), 30255 [UPDATE], 99thPercentileLatency(us), 41855 [UPDATE], Return=OK, 1250212 [SCAN], Operations, 1250644 [SCAN], AverageLatency(us), 33153.01709839091 [SCAN], MinLatency(us), 742 [SCAN], MaxLatency(us), 645119 [SCAN], 95thPercentileLatency(us), 62879 [SCAN], 99thPercentileLatency(us), 80447 [SCAN], Return=OK, 1250644 |
In the metrics screenshot, the first interval is the performance of RWQueueRpcExecutor and the second interval is FastPathRWQueueRpcExecutor.
Attachments
Attachments
Issue Links
- is a clone of
-
HBASE-17808 FastPath for RWQueueRpcExecutor
-
- Resolved
-
- links to