Uploaded image for project: 'Mesos'
  1. Mesos
  2. MESOS-8668

Transition libprocess on Windows to use the Thread Pool API

    XMLWordPrintableJSON

Details

    • Epic
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.7.0
    • libprocess
    • Windows 10

    • libprocess-iocp

    Description

      This epic covers the work necessary to use the Windows Thread Pool API instead of libev or libevent on Windows.

      The design of libprocess closely mimics the built-in Thread Pool API, using I/O Completion Ports for asynchronous I/O. It makes sense to use this API because the semantics of libprocess map 1-1 to the Windows async asynchronous support.

      Transitioning to IOCP is necessary to implement non-blocking pipes, as the existing eventing libraries (e.g. libevent) only implement asynchronous sockets on Windows; but there are many instances of Mesos requiring asynchronous pipes too. As such, the existing use of libevent is fundamentally broken.

      Specifically, this was first discovered in test HungDockerTest.ROOT_DOCKER_InspectHungDuringPull, but is also evidenced through IOTest.Read and others which use non-blocking pipes.

      The alternative eventing libraries libev and libuv were considered, but we are choosing not to use them as they do not implement non-blocking pipes sufficiently for Windows. libev is not easy to build on Windows, and does not support IOCP. While libevent uses IOCP for reading and writing sockets, it uses select() to detect and dispatch events (which results in poor performance), and does not use support IOCP for pipes. libuv claims to support IOCP on Windows, but they do not support named pipes and only supportes synchronous pipes. Moreover, it is simpler to use the Thread Pool API in libprocess directly, instead of using a library to abstract over that API, as libprocess already provides the abstraction we're seeking for Mesos.

      Additionally, by using the built-in Windows IOCP support, we can eventually support SSL using the Windows crypto libraries, instead of requiring OpenSSL on Windows (the current case for libprocess).

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              andschwa Andrew Schwartzmeyer
              Andrew Schwartzmeyer Andrew Schwartzmeyer
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - 1,008h
                  1,008h
                  Remaining:
                  Remaining Estimate - 1,008h
                  1,008h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified