Details
-
Improvement
-
Status: Resolved
-
Trivial
-
Resolution: Duplicate
-
2.4.0
-
None
Description
Seperated node selector config option between executors and driver.
This removes the spark.kubernetes.node.selector config option and seperates it to
spark.kubernetes.driver.selector and spark.kubernetes.executor.selector
to allow seperate node selectors on drivers and executors.
My personal use case for this change is that on AWS we have cheap spotinstances that can terminate at any moment which is okay for executors but not the driver.
With a single node selector option I am unable to use the spot instances to save costs on the executor nodes.
Attachments
Issue Links
- duplicates
-
SPARK-24434 Support user-specified driver and executor pod templates
- Resolved
- links to