Uploaded image for project: 'Kudu'
  1. Kudu
  2. KUDU-2776

Java AUTO_FLUSH_BACKGROUND behaves like "AUTO_FLUSH_FOREGROUND" when tablet locations are cached

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 1.9.0
    • None
    • None

    Description

      The above piece of a Java flamegraph shows the main application thread apply}}ing operations to a {{KuduSession in AUTO_FLUSH_BACKGROUND mode. The doFlush call is meant to set up a callbacks that actually send the rows to Kudu and that are triggered by the fulfillment of a Deferred for the tablet locations. However, when the tablet locations are cached, this Deferred can be fulfilled basically instantly, so by the time that the apply}}ing thread calls {{addCallbacks, the deferred is complete, and as an optimization the async library executes the callbacks on the thread that is adding them to the fulfilled Deferred. This means the apply}}ing thread executes the code that sends rows to Kudu, which is the opposite of how {{AUTO_FLUSH_BACKGROUND is meant to work.

      We lose out on the ability to have multiple batches in flight at once, and we serialize the application logic, apply, and actually sending rows to Kudu.

      Attachments

        1. image-2019-04-16-14-37-07-908.png
          271 kB
          William Berkeley

        Activity

          People

            Unassigned Unassigned
            wdberkeley William Berkeley
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: