Details
-
Bug
-
Status: Open
-
Critical
-
Resolution: Unresolved
-
0.8.0
-
None
-
None
-
zeppelin 0.8.0
spark 2.1.0
Description
spark interpreter always writing to /tmp file system, create a sparkXXXX directory, Space used keeps increasing after every %spark interpreter execute.
I had config spark.local.dir = [other path] in interpreter configuration or add SPARK_SUBMIT_OPTIONS in zeppelin-env.sh, but no effect.