Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-13291

Lift the scan ceiling

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Implemented
    • 1.0.0
    • None
    • Scanners
    • None

    Description

      Scanning medium sized rows with multiple concurrent scanners exhibits interesting 'ceiling' properties. A server runs at about 6.7k ops a second using 450% of possible 1600% of CPUs when 4 clients each with 10 threads doing scan 1000 rows. If I add '--filterAll' argument (do not return results), then we run at 1450% of possible 1600% possible but we do 8k ops a second.

      Let me attach flame graphs for two cases. Unfortunately, there is some frustrating dark art going on. Let me try figure it... Filing issue in meantime to keep score in.

      Attachments

        1. traces.smaller.svg
          397 kB
          Michael Stack
        2. traces.small2.svg
          367 kB
          Michael Stack
        3. traces.nofilter.svg
          563 kB
          Michael Stack
        4. traces.filterall.svg
          341 kB
          Michael Stack
        5. traces.7.svg
          338 kB
          Michael Stack
        6. TimeRange.patch
          0.6 kB
          Anoop Sam John
        7. Screen Shot 2015-03-26 at 3.39.33 PM.png
          573 kB
          Michael Stack
        8. Screen Shot 2015-03-26 at 12.12.13 PM.png
          100 kB
          Michael Stack
        9. scan_no_mvcc_optimized.svg
          354 kB
          Michael Stack
        10. q (1).png
          10 kB
          Michael Stack
        11. nonBBposAndInineMvccVint.txt
          4 kB
          Michael Stack
        12. hack_to_bypass_bb.txt
          4 kB
          Michael Stack
        13. 13291.inlining.txt
          7 kB
          Michael Stack
        14. 13291.hacks.txt
          25 kB
          Michael Stack

        Issue Links

          Activity

            People

              stack Michael Stack
              stack Michael Stack
              Votes:
              0 Vote for this issue
              Watchers:
              20 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: