Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-28256

Enhance ByteBufferUtils.readVLong to read more bytes at a time

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.6.0, 2.5.8, 3.0.0-beta-2
    • Performance
    • None

    Description

      Currently, ByteBufferUtils.readVLong is used to decode rows in all data block encodings in order to read the memstoreTs field. For a data block encoding like prefix, ByteBufferUtils.readVLong can surprisingly occupy over 50% of the CPU time in BufferedEncodedSeeker.decodeNext (which can be quite a hot method in seek operations).

       

      Since memstoreTs will typically require at least 6 bytes to store, we could look to vectorize the read path for readVLong to read 8 bytes at a time instead of a single byte at a time (like in https://issues.apache.org/jira/browse/HBASE-28025) in order to increase performance.

       

      Attached is a CPU flamegraph of a region server process which shows that we spend a surprising amount of time in decoding rows from the DBE in ByteBufferUtils.readVLong.

      Attachments

        1. ReadVLongBenchmark.zip
          2 kB
          Becker Ewing
        2. async-prof-rs-cpu.html
          411 kB
          Becker Ewing

        Issue Links

          Activity

            People

              bewing Becker Ewing
              bewing Becker Ewing
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: