Details
-
Bug
-
Status: Open
-
Minor
-
Resolution: Unresolved
-
3.5.0
-
None
-
None
Description
I noticed an error in my Driver log:
org.apache.spark.rpc.RpcTimeoutException: Cannot receive any reply from /XXX.YYY.ZZZ.WWW:35560 in 10 seconds. This timeout is controlled by spark.rpc.askTimeout
The error message and indeed the Driver invocation command references a timeout value of 10 seconds.
However the Spark documentation implies that the default value for spark.rpc.askTimeout should be 120 seconds (indirectly from spark.network.timeout)
Ref: https://spark.apache.org/docs/3.5.0/configuration.html#networking
I can see the problem from the code here: https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/Client.scala#L285