Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-20601

Add multiPut support and other miscellaneous to PE

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.0.0
    • 3.0.0-alpha-1, 2.1.0
    • tooling
    • None
    • Hide
      1. Add multiPut support
      Set --multiPut=number to enable batchput(meanwhile, --autoflush need be set to false)

      2. Add Connection Count support
      Added a new parameter connCount to PE. set --connCount=2 means all threads will share 2 connections.
      oneCon option and connCount option shouldn't be set at the same time.

      3. Add avg RT and avg TPS/QPS statstic for all threads

      4. Delete some redundant code
      Now RandomWriteTest is inherited from SequentialWrite.
      Show
      1. Add multiPut support Set --multiPut=number to enable batchput(meanwhile, --autoflush need be set to false) 2. Add Connection Count support Added a new parameter connCount to PE. set --connCount=2 means all threads will share 2 connections. oneCon option and connCount option shouldn't be set at the same time. 3. Add avg RT and avg TPS/QPS statstic for all threads 4. Delete some redundant code Now RandomWriteTest is inherited from SequentialWrite.

    Description

      Add some useful stuff and some refinement to PE tool
      1. Add multiPut support
      Though we have BufferedMutator, sometimes we need to benchmark batch put in a certain number.
      Set --multiPut=number to enable batchput(meanwhile, --autoflush need be set to false)

      2. Add Connection Number support
      Before, there is only on parameter to control the connection used by threads. oneCon=true means all threads use one connection, false means each thread has it own connection.
      When thread number is high and oneCon=false, we noticed high context switch frequency in the machine which PE run on, disturbing the benchmark results(each connection has its own netty worker threads, 2*CPU IIRC).
      So, added a new parameter connCount to PE. set --connCount=2 means all threads will share 2 connections.

      3. Add avg RT and avg TPS/QPS statstic for all threads
      Useful when we want to meansure the total throughtput of the cluster

      4. Delete some redundant code
      Now RandomWriteTest is inherited from SequentialWrite.

      Attachments

        1. HBASE-20601.branch-2.006.patch
          33 kB
          Allan Yang
        2. HBASE-20601.003.patch
          33 kB
          Allan Yang
        3. HBASE-20601.002.patch
          33 kB
          Allan Yang
        4. HBASE-20601.branch-2.005.patch
          33 kB
          Allan Yang
        5. HBASE-20601.patch
          24 kB
          Allan Yang
        6. HBASE-20601.branch-2.004.patch
          31 kB
          Allan Yang
        7. HBASE-20601.branch-2.003.patch
          31 kB
          Allan Yang
        8. HBASE-20601.branch-2.002.patch
          30 kB
          Allan Yang
        9. HBASE-20601.branch-2.patch
          30 kB
          Allan Yang

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            allan163 Allan Yang
            allan163 Allan Yang
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment