Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-20188

Evaluate and address performance delta between branch-1 and branch-2

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Umbrella
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • None
    • None
    • Performance
    • None

    Description

      How does 2.0.0 compare to old versions? Is it faster, slower? There is rumor that it is much slower, that the problem is the asyncwal writing. Does in-memory compaction slow us down or speed us up? What happens when you enable offheaping?

      Keep notes here in this umbrella issue. Need to be able to say something about perf when 2.0.0 ships.

      Attachments

        1. CAM-CONFIG-V01.patch
          3 kB
          Anastasia Braginsky
        2. flamegraph-1072.1.svg
          1020 kB
          Michael Stack
        3. flamegraph-1072.2.svg
          974 kB
          Michael Stack
        4. HBase 2.0 performance evaluation - 8GB.pdf
          36 kB
          Eshcar Hillel
        5. HBase 2.0 performance evaluation - 8GB(1).pdf
          58 kB
          Eshcar Hillel
        6. HBase 2.0 performance evaluation - Basic vs None_ system settings.pdf
          36 kB
          Eshcar Hillel
        7. HBase 2.0 performance evaluation - throughput SSD_HDD.pdf
          49 kB
          Eshcar Hillel
        8. HBASE-20188.sh
          2 kB
          Eshcar Hillel
        9. HBASE-20188-xac.sh
          2 kB
          Eshcar Hillel
        10. hbase-env.sh
          8 kB
          Michael Stack
        11. hbase-site.xml
          3 kB
          Eshcar Hillel
        12. hbase-site.xml
          8 kB
          Michael Stack
        13. hits_with_fp_scheduler.png
          33 kB
          Michael Stack
        14. hits.png
          48 kB
          Michael Stack
        15. ITBLL2.5B_1.2.7vs2.0.0_cpu.png
          165 kB
          Michael Stack
        16. ITBLL2.5B_1.2.7vs2.0.0_gctime.png
          77 kB
          Michael Stack
        17. ITBLL2.5B_1.2.7vs2.0.0_iops.png
          40 kB
          Michael Stack
        18. ITBLL2.5B_1.2.7vs2.0.0_load.png
          112 kB
          Michael Stack
        19. ITBLL2.5B_1.2.7vs2.0.0_memheap.png
          81 kB
          Michael Stack
        20. ITBLL2.5B_1.2.7vs2.0.0_memstore.png
          23 kB
          Michael Stack
        21. ITBLL2.5B_1.2.7vs2.0.0_ops_NOT_summing_regions.png
          91 kB
          Michael Stack
        22. ITBLL2.5B_1.2.7vs2.0.0_ops.png
          56 kB
          Michael Stack
        23. lock.127.workloadc.20180402T200918Z.svg
          63 kB
          Michael Stack
        24. lock.2.memsize2.c.20180403T160257Z.svg
          99 kB
          Michael Stack
        25. perregion.png
          48 kB
          Michael Stack
        26. run_ycsb.sh
          7 kB
          Michael Stack
        27. total.png
          23 kB
          Michael Stack
        28. tree.txt
          985 kB
          Michael Stack
        29. workloadx
          3 kB
          Eshcar Hillel
        30. workloadx
          3 kB
          Eshcar Hillel
        31. YCSB_CPU.png
          46 kB
          Michael Stack
        32. YCSB_GC_TIME.png
          24 kB
          Michael Stack
        33. YCSB_IN_MEMORY_COMPACTION=NONE.ops.png
          31 kB
          Michael Stack
        34. YCSB_in-memory-compaction=NONE.ops.png
          28 kB
          Michael Stack
        35. YCSB_load.png
          42 kB
          Michael Stack
        36. YCSB_MEMSTORE.png
          29 kB
          Michael Stack
        37. YCSB_OPs.png
          32 kB
          Michael Stack

        Issue Links

        contains

        Sub-task - The sub-task of the issue HBASE-20710 extra cloneFamily() in Mutation.add(Cell)

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Resolved
        is related to

        Bug - A problem which impairs or prevents the functions of the product. HBASE-21657 PrivateCellUtil#estimatedSerializedSizeOf has been the bottleneck in 100% scan case.

        • Major - Major loss of function.
        • Resolved

        Improvement - An improvement or enhancement to an existing feature or task. HBASE-21879 Read HFile's block to ByteBuffer directly instead of to byte for reducing young gc purpose

        • Major - Major loss of function.
        • Closed
        relates to

        Sub-task - The sub-task of the issue HBASE-20337 Update the doc on how to setup shortcircuit reads; its stale

        • Major - Major loss of function.
        • Resolved

        Bug - A problem which impairs or prevents the functions of the product. HBASE-20411 Ameliorate MutableSegment synchronize

        • Major - Major loss of function.
        • Resolved

        Bug - A problem which impairs or prevents the functions of the product. HBASE-20457 Return immediately for a scan rpc call when we want to switch from pread to stream

        • Major - Major loss of function.
        • Resolved
        links to

        Web Link 1.2.7 vs 2.0.0 Flush History during YCSB Load using Client-Side Batching

        Web Link hbase1 to hbase2 compare with and without IN_MEMORY_COMPACTION and with and without MSLAB

        Web Link Profling log/notes

        Web Link Review Board (branch-2.0)

        1.
        		 [locking] Write-time worst offenders Sub-task Resolved Unassigned Actions
        2.
        		 Update the doc on how to setup shortcircuit reads; its stale Sub-task Resolved Michael Stack Actions
        3.
        		 IMC Default Parameters for 2.0.0 Sub-task Resolved Eshcar Hillel Actions
        4.
        		 [DOC] Add note on perf to upgrade section Sub-task Resolved Michael Stack Actions
        5.
        		 [IHC] Workloadx Sub-task Resolved Unassigned Actions
        6.
        		 [PERFORMANCE] Flushing is 2x slower in hbase2. Sub-task Resolved Michael Stack Actions
        7.
        		 Better heap utilization for IMC with MSLABs Sub-task Resolved Eshcar Hillel Actions
        8.
        		 Tighter ByteBufferKeyValue Cell Comparator Sub-task Resolved Michael Stack Actions
        9.
        		 Tighter ByteBufferKeyValue Cell Comparator; part 2 Sub-task Resolved Michael Stack Actions
        10.
        		 SegmentScanner does over-comparing when one flushing Sub-task Resolved Michael Stack Actions
        11.
        		 extra cloneFamily() in Mutation.add(Cell) Sub-task Resolved Hua Xiang Actions
        12.
        		 ServerRpcConnection logging cleanup Sub-task Resolved Michael Stack Actions
        13.
        		 Save recalculating families in a WALEdit batch of Cells Sub-task Resolved Michael Stack Actions
        14.
        		 MemStoreLABImp::copyIntoCell uses 7% CPU when writing Sub-task Resolved Michael Stack Actions
        15.
        		 Unsafe access cleanup Sub-task Resolved Sahil Aggarwal Actions
        16.
        		 Remove all the CSLM#size operation in our memstore because it's an quite time consuming. Sub-task Resolved Zheng Hu Actions
        17.
        		 Most of KeyValueUtil#length can be replaced by cell#getSerializedSize for better performance because the latter one has been optimized Sub-task Resolved Zheng Hu Actions
        18.
        		 Size of in-memory compaction thread pool should be configurable Sub-task Resolved Zheng Hu Actions

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            stack Michael Stack
            Votes:
            0 Vote for this issue
            Watchers:
            33 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment