Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
0.6.2, 0.7.0, 0.7.1, 0.8.0
-
None
-
macOS Sierra (OS X) Version 10.12.3, Java 1.7 to Java 1.8 (1.7u80, 1.8u121), Maven 3.2.5 or 3.3.9
Description
The issue is a follow-up of https://issues.apache.org/jira/browse/ZEPPELIN-2370 - clarifying the reasons.
When the CLASSPATH variable is exported in bash (even if it is empty), some zeppelin interpreters fail (on OS X). The error occurs if you run
export CLASSPATH= ./bin/zeppelin.sh
and then try to execute the md interpreter, e.g. via
%md *Test*
I then get the following error
java.lang.IncompatibleClassChangeError: class org.objectweb.asm.tree.ClassNode has interface org.objectweb.asm.ClassVisitor as super class at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:800) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) at java.net.URLClassLoader.access$100(URLClassLoader.java:71) at java.net.URLClassLoader$1.run(URLClassLoader.java:361) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:800) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) at java.net.URLClassLoader.access$100(URLClassLoader.java:71) at java.net.URLClassLoader$1.run(URLClassLoader.java:361) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at org.parboiled.transform.ParserTransformer.extendParserClass(ParserTransformer.java:43) at org.parboiled.transform.ParserTransformer.transformParser(ParserTransformer.java:38) at org.parboiled.Parboiled.createParser(Parboiled.java:54) at org.pegdown.plugins.PegDownPlugins$Builder.withPlugin(PegDownPlugins.java:126) at org.apache.zeppelin.markdown.PegdownParser.<init>(PegdownParser.java:35) at org.apache.zeppelin.markdown.Markdown.createMarkdownParser(Markdown.java:75) at org.apache.zeppelin.markdown.Markdown.open(Markdown.java:85) at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:70) at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:483) at org.apache.zeppelin.scheduler.Job.run(Job.java:175) at org.apache.zeppelin.scheduler.ParallelScheduler$JobRunner.run(ParallelScheduler.java:162) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745)
A further investigation showed that setting CLASSPATH will change the classpath of an interpreter. Running the following Scala code in Zeppelin prints the classpath of the Scala interpreter (which could/should differ from the classpath of the Zeppelin server):
def urlses(cl: ClassLoader): Array[java.net.URL] = cl match { case null => Array() case u: java.net.URLClassLoader => u.getURLs() ++ urlses(cl.getParent) case _ => urlses(cl.getParent) } val urls = urlses(getClass.getClassLoader) println(urls.mkString("\n"))
If CLASSPATH is set, then we find
zeppelin-0.7.1-bin-all/lib/asm-3.1.jar
in the interpreter classpath (and many other jars in zeppelin-0.7.1-bin-all/lib/).
However, if CLASSPATH is unset, then only jars from zeppelin-0.7.1-bin-all/lib/interpreter/ are found in the interpreters classpath. To check this, stop zeppelin and run
unset CLASSPATH ./bin/zeppelin.sh
and repeat the tests above.
EXPLANATION / FIX
The issue is caused by zeppelin.sh modifying the CLASSPATH variable which may be exported to interpreters.sh, which in turn modifies and uses the CLASSPATH variable. Modification of the CLASSPATH variable inside the sh scripts may cause side effects. Instead I would recommend leaving CLASSPATH unchanged.
I am providing a pull request on GitHub.
Attachments
Issue Links
- links to
