Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Incomplete
-
2.3.0
-
None
Description
SPARK-20898 added a new configuration to blacklist a node/executor on fetch
failures. This config was deemed risky at the time and was disabled by default
until more data is collected.
This commit aims to enable that feature by default as we have seen couple of
instances where that feature was found to be useful. The failures occur because of issues with node manager and external shuffle service on that node.
Attachments
Issue Links
- relates to
-
SPARK-20898 spark.blacklist.killBlacklistedExecutors doesn't work in YARN
- Resolved
- links to