Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
3.1.0
-
None
Description
SPARK-23257 adds kerberized hdfs support for Spark on K8s. A major addition needed is the support for proxy user. A proxy user is impersonated by a superuser who executes operations on behalf of the proxy user. More on this:
https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Superusers.html
https://github.com/spark-notebook/spark-notebook/blob/master/docs/proxyuser_impersonation.md
This has been implemented for Yarn upstream and Spark on Mesos here:
https://github.com/mesosphere/spark/pull/26
ifilonenko creating this issue according to our discussion.
Attachments
Attachments
Issue Links
- links to