Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
3.1.0
-
None
-
None
Description
Currently, if I specify a default filesystem:
fs.defaultFS=hdfs://host
And a scheme-less event log directory:
spark.eventLog.dir=/some/dir
Then the event log directory is always resolved with respect to the local filesystem.
I propose that it should be resolved with respect to the default filesystem.
Targeting Spark 3.0.0 since this is technically a breaking change.
Attachments
Issue Links
- links to