Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
1.6.0, 2.0.0, 2.1.0, 2.2.0, 2.3.0, 2.4.0
-
None
Description
In Standalone, while launching executor during ExecutorRunner.start() is always failed in Worker, Master will continue to launch new executor for the same Worker indefinitely.
The issue is easy to reproduce by running a unit test with local-cluster mode and set a wrong spark.test.home(e.g. /tmp). Then, when running unit test, test would get stuck and we can see endless executor directories under /tmp/work/app/.