Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Incomplete
-
2.4.5
-
None
Description
Please change the deployment script for the workers to support one Shared Volume with the driver. E.g. a possible implementation:
https://kubernetes.io/docs/tasks/access-application-cluster/communicate-containers-same-pod-shared-volume/
Atm. If a driver runs in Kubernetes it is not possible to share a volume between the driver and the workers.
Enabling a shared volume between Driver and Workers would allow the use of sc.textFile(file:///path/file.txt) to be used without any need of HDFS or S3 Storage (Which in fact is a huge benefit)