Uploaded image for project: 'Mesos'
  1. Mesos
  2. MESOS-9336

Hook interfaces and implementation performs unnecessary protobuf copying.

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Accepted
    • Major
    • Resolution: Unresolved
    • None
    • None
    • modules

    Description

      The interfaces and implementation of hooks perform significant protobuf copying and would benefit from copy elimination. Consider the case of the master task label decorator:

      https://github.com/apache/mesos/blob/1.7.0/src/hook/manager.cpp#L108-L138

      Labels HookManager::masterLaunchTaskLabelDecorator(
          const TaskInfo& taskInfo,
          const FrameworkInfo& frameworkInfo,
          const SlaveInfo& slaveInfo)
      {
        synchronized (mutex) {
          // We need a mutable copy of the task info and set the new
          // labels after each hook invocation. Otherwise, the last hook
          // will be the only effective hook setting the labels.
          TaskInfo taskInfo_ = taskInfo;
      
          foreachpair (const string& name, Hook* hook, availableHooks) {
            const Result<Labels> result =
              hook->masterLaunchTaskLabelDecorator(
                  taskInfo_,
                  frameworkInfo,
                  slaveInfo);
      
            // NOTE: If the hook returns None(), the task labels won't be
            // changed.
            if (result.isSome()) {
              taskInfo_.mutable_labels()->CopyFrom(result.get());
            } else if (result.isError()) {
              LOG(WARNING) << "Master label decorator hook failed for module '"
                          << name << "': " << result.error();
            }
          }
      
          return taskInfo_.labels();
        }
      }
      

      https://github.com/apache/mesos/blob/1.7.0/src/master/master.cpp#L5619-L5634

              foreach (
                  TaskInfo& task, *message.mutable_task_group()->mutable_tasks()) {
                taskIds.insert(task.task_id());
                totalResources += task.resources();
      
                addTask(task, framework, slave);
      
                if (HookManager::hooksAvailable()) {
                  // Set labels retrieved from label-decorator hooks.
                  task.mutable_labels()->CopyFrom(
                      HookManager::masterLaunchTaskLabelDecorator(
                          task,
                          framework->info,
                          slave->info));
                }
              }
      
      1. We copy the entire task info.
      2. We copy the labels of the task info twice for each label decorator hook (the hook itself has to copy all existing ones since the API will re-write the labels, and then we copy the result back into the task info).
      3. We copy the labels back out to the caller.
      4. The master then copies the labels back into the task.

      In the case of 1 hook, that's 1 copy of entire the task info (including labels) and 4 copies of only the labels.

      Without changing the interface, we can eliminate 3 of the label copying cases, and we would be left with 1 copy of the entire task info (including labels), and 1 copy of the labels (per hook).

      With changing the interface, we can achieve 0 copies. This can be done by making it a mutation interface, or making it truly a label decorator (labels can only be added) with the modification that hooks don't get to see the additional labels added by previous hooks. These would all be breaking changes that require adjusting hooks.

      Attachments

        Activity

          People

            Unassigned Unassigned
            bmahler Benjamin Mahler
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: