Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-47317

taskSummary API should include the task state as a parameter

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.5.1
    • None
    • Spark Core, Web UI
    • None

    Description

      After the JIRA, https://issues.apache.org/jira/browse/SPARK-26119, task summary only contain successful task.

      In our scenario, utilize the REST API [/applications/[app-id]/stages/[stage-id]/[stage-attempt-id]/taskSummary] to obtain the taskSummary metric. Use the Max(schedule delay) to diagnose why the stage duration is excessively long. but in some case the slowest task is a failed state task, so task summary metrics is not included this task.

      expect：

      the rest api [/applications/[app-id]/stages/[stage-id]/[stage-attempt-id]/taskSummary] task state as a parameter.

      Attachments

        Activity

          People

            Unassigned Unassigned
            xumanbu jam xu
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: