Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
Description
As Spark recommends use spark-submit to run spark application, Zeppelin can have an option to run spark interpreter process using spark-submit, after ZEPPELIN-160.
basic idea is, introduce environment variable 'USE_SPARK_SUBMIT'.
When SPARK_HOME and USE_SPARK_SUBMIT is defined, that interpreter.sh launches spark interpreter process using spark-submit.
This could help remove some problems oriented from Zeppelin starts SparkContext not exactly the same way spark-submit does.