Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-20601

Add multiPut support and other miscellaneous to PE

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.0.0
    • 3.0.0-alpha-1, 2.1.0
    • tooling
    • None
    • Hide
      1. Add multiPut support
      Set --multiPut=number to enable batchput(meanwhile, --autoflush need be set to false)

      2. Add Connection Count support
      Added a new parameter connCount to PE. set --connCount=2 means all threads will share 2 connections.
      oneCon option and connCount option shouldn't be set at the same time.

      3. Add avg RT and avg TPS/QPS statstic for all threads

      4. Delete some redundant code
      Now RandomWriteTest is inherited from SequentialWrite.
      Show
      1. Add multiPut support Set --multiPut=number to enable batchput(meanwhile, --autoflush need be set to false) 2. Add Connection Count support Added a new parameter connCount to PE. set --connCount=2 means all threads will share 2 connections. oneCon option and connCount option shouldn't be set at the same time. 3. Add avg RT and avg TPS/QPS statstic for all threads 4. Delete some redundant code Now RandomWriteTest is inherited from SequentialWrite.

    Description

      Add some useful stuff and some refinement to PE tool
      1. Add multiPut support
      Though we have BufferedMutator, sometimes we need to benchmark batch put in a certain number.
      Set --multiPut=number to enable batchput(meanwhile, --autoflush need be set to false)

      2. Add Connection Number support
      Before, there is only on parameter to control the connection used by threads. oneCon=true means all threads use one connection, false means each thread has it own connection.
      When thread number is high and oneCon=false, we noticed high context switch frequency in the machine which PE run on, disturbing the benchmark results(each connection has its own netty worker threads, 2*CPU IIRC).
      So, added a new parameter connCount to PE. set --connCount=2 means all threads will share 2 connections.

      3. Add avg RT and avg TPS/QPS statstic for all threads
      Useful when we want to meansure the total throughtput of the cluster

      4. Delete some redundant code
      Now RandomWriteTest is inherited from SequentialWrite.

      Attachments

        1. HBASE-20601.patch
          24 kB
          Allan Yang
        2. HBASE-20601.branch-2.patch
          30 kB
          Allan Yang
        3. HBASE-20601.branch-2.006.patch
          33 kB
          Allan Yang
        4. HBASE-20601.branch-2.005.patch
          33 kB
          Allan Yang
        5. HBASE-20601.branch-2.004.patch
          31 kB
          Allan Yang
        6. HBASE-20601.branch-2.003.patch
          31 kB
          Allan Yang
        7. HBASE-20601.branch-2.002.patch
          30 kB
          Allan Yang
        8. HBASE-20601.003.patch
          33 kB
          Allan Yang
        9. HBASE-20601.002.patch
          33 kB
          Allan Yang

        Issue Links

          Activity

            People

              allan163 Allan Yang
              allan163 Allan Yang
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: