Uploaded image for project: 'Mesos'
  1. Mesos
  2. MESOS-7398

HierarchicalAllocatorProcess::allocatable makes strong assumptions about both resource providers and users.

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      The function HierarchicalAllocatorProcess::allocatable is used in the allocator to decide whether a set of resources will be considered when calculating offers. It currently hardcodes minimal requirements for a number of common resource kinds.

      While it seems to have in the past be used to enforce the offer side of minimal task resources by not offering resources which we didn't want to be used for tasks, it now seems to mainly help to minimize performance overhead from too many small offers (instead too small resource amounts are kept out of the offer pool until they became accumulated into larger resources).

      While allocatable has already in the past prevented allocating sets of only certain resource kinds (e.g., a Resources holding only GPU is not allocatable; the same holds for custom resource kinds), the current approach breaks down with the introduction of resource providers with MESOS-7235 which might provide a single kind of resource each and which in the case of external resource providers might never "reside" on the same agent as e.g., CPU.

      It seems that we need to separate the different concerns of allocatable into dedicated functions, and adjust it to remain useful in a world of (external) resource providers.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            bbannier Benjamin Bannier

            Dates

              Created:
              Updated:

              Slack

                Issue deployment