Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-12062

Master rebuilding historical SparkUI should be asynchronous

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.0.0
    • 1.6.0, 2.0.0
    • Deploy
    • None

    Description

      When a long-running application finishes, it takes a while (sometimes minutes) to rebuild the SparkUI. However, in Master.scala this is currently done within the RPC event loop, which runs only in 1 thread. Thus, in the mean time no other applications can register with this master.

      Attachments

        Issue Links

          Activity

            People

              bryanc Bryan Cutler
              andrewor14 Andrew Or
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: