Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-30582

Spark UI is not showing Aggregated Metrics by Executor in stage page

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 3.0.0
    • 3.0.0
    • Web UI
    • None

    Description

      There are scenarios where Spark History Server is located behind the VPC. So whenever api calls hit to get the executor Summary(allexecutors). There can be delay in getting the response of executor summary and in mean time "stage-page-template.html" is loaded and the response of executor Summary is not added to the stage-page-template.html.

      As the result of which Aggregated Metrics by Executor in stage page is showing blank.

      This scenario can be easily found in the cases when there is some proxy-server which is responsible for sending the request and response to spark History server.
      This can be reproduced in Knox/In-house proxy servers which are used to send and receive response to Spark History Server.

      Alternative scenario to test this case, Open the spark UI in developer mode in browser add some breakpoint in stagepage.js, this will add some delay in getting the response and now if we check the spark UI for stage Aggregated Metrics by Executor in stage page is showing blank.

      Attachments

        1. SparkUIStagePage.mov
          18.93 MB
          Saurabh Chawla

        Issue Links

          Activity

            People

              saurabhc100 Saurabh Chawla
              saurabhc100 Saurabh Chawla
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: