Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5169

Parallelise read I/O of BufferPool::Pin()

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • Impala 2.9.0
    • Impala 2.9.0
    • Backend
    • None

    Description

      Currently read I/O in BufferPool is synchronous. In some cases this can lead to poor resource utilisation and I/O throughput, because:

      • We don't dispatch parallel reads to multiple scratch disks or high-throughput SSDs
      • Issuing reads of contiguous scratch ranges at the same time improves the odds that the second read can be served without a disk seek or by the disks internal cache.
      • Expose a batched Pin() interface that can pin multiple buffers at the same time
      • Expose an asynchronous Pin() interface that can start the read, and allow the client to wait for it.

      The first alternative is probably simplest.

      Attachments

        Issue Links

          Activity

            People

              tarmstrong Tim Armstrong
              tarmstrong Tim Armstrong
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: