Uploaded image for project: 'Mesos'
  1. Mesos
  2. MESOS-8517

Master http endpoint /tasks should include "executor/directory" field

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Abandoned
    • 1.4.1
    • None
    • HTTP API
    • None

    Description

      Hello all,

      I noticed that the Task information returned by a mesos agent is different from the task information returned by the mesos master.

      Agent's /state endpoint, when returning each task, includes the directory field (inside the executors array), which is super useful if we want to browse this task's sandbox files. But at the same time, the agent HTTP API does not provide a way to get a task by its ID.

      So we need to talk to the Mesos master to get a task by its ID and we can do it using the /tasks?task_id=<task_id endpoint. But here we do not have any information about this task's executor.

      So given a TaskID, to be able to download/browse its sandbox files we need 3+ API calls:

      • One to the master (/tasks) to get the task's JSON;
      • Anoter to the master (/slaves?slave_id=<slave-id> to get the slave IP address;
      • Another to this slave (//state) to get all its tasks. Then we iterate the lists of tasks (of each executor) searching for our TaskID; get the directory field.
      • And then, the final API call, made on endpoint (/files/browse?path=) to get the contents of the required file

      I would like to know if it's possible to the Master to include this directory field when returning info about a specific task.

      Also, is there a easier way to do what I described here? GIven a TaskID, get the content of a file inside its sandbox (stdout/stderr would be the most common options)

      Thanks a lot.

      Here is one example of each JSON returned (only the important parts)

      Master /tasks?task_id=<task_id> endpoint

      {
         "tasks" : [
            {
               "resources" : {
                  "disk" : 0,
                  "gpus" : 0,
                  "ports" : "[31720-31720]",
                  "cpus" : 0.3,
                  "mem" : 256
               },
               "executor_id" : "",
               "labels" : [
               ],
               "framework_id" : "c7fe8caf-3772-4570-bb36-149d600053af-0000",
               "id" : "",
               "slave_id" : "78a58d98-f011-41bf-b5d7-a92700bcda93-S20847",
               "discovery" : { },
               "role" : "*",
               "state" : "TASK_RUNNING"
            }
         ]
      }
      

      Slave's /state endpoint

      {
         "build_user" : "centos",
         "frameworks" : [
            {
               "role" : "*",
               "completed_executors" : ,
               "executors" : [
                  {
                     "tasks" : [ ],
                     "directory" : "/tmp/mesos/slaves/78a58d98-f011-41bf-b5d7-a92700bcda93-S20847/frameworks/c7fe8caf-3772-4570-bb36-149d600053af-0000/executors/task_name.447d755d-0616-11e8-96d4-ee61be2ee5b4/runs/14f119de-708c-4978-b04e-60886a12be57",
                     "container" : "14f119de-708c-4978-b04e-60886a12be57",
                     "queued_tasks" : [],
                     "resources" : {
                        "cpus" : 0.4,
                        "mem" : 288,
                        "ports" : "[31720-31720]",
                        "disk" : 0,
                        "gpus" : 0
                     },
                     "completed_tasks" : [],
                     "labels" : [ ],
                     "role" : "*"
                  }
               ],
               "id" : "c7fe8caf-3772-4570-bb36-149d600053af-0000",
               "checkpoint" : true
            }
         ]
      }
      
      

      Thanks,

      Attachments

        Activity

          People

            Unassigned Unassigned
            daltonmatos Dalton Matos Coelho Barreto
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: