Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-39222

Add JvmPauseMonitor to Spark History Server

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.0.0, 2.4.8, 3.0.0, 3.2.1
    • None
    • Spark Core

    Description

      It is hard to understand what happens with Spark History Server when long JVM pauses (GC or host level pauses) are causing unresponsiveness.
      Similar to Hadoop's implementation (initially in HADOOP-9618) it would be beneficial to add the JVMPauseMonitor to the HistoryServer (HistoryServerSuite.scala).
      This will make GC pauses obvious in logs and lets administrators easily notice it and react in time - adjust configurations with increasing the SHS heap size.

      Attachments

        Activity

          People

            Unassigned Unassigned
            mszurap Miklos Szurap
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: