Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
2.0.0, 2.4.8, 3.0.0, 3.2.1
-
None
Description
It is hard to understand what happens with Spark History Server when long JVM pauses (GC or host level pauses) are causing unresponsiveness.
Similar to Hadoop's implementation (initially in HADOOP-9618) it would be beneficial to add the JVMPauseMonitor to the HistoryServer (HistoryServerSuite.scala).
This will make GC pauses obvious in logs and lets administrators easily notice it and react in time - adjust configurations with increasing the SHS heap size.