Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
0.8.1
-
None
-
None
Description
I just downloaded and installed the netinst binary package of zeppelin 0.8.1 and installed the shell interpreter with the command:
./bin/install-interpreter.sh --name shell
If I open up a new note and try to run a shell command
%sh ls -lh
I get the following error.
INFO [2019-08-21 13:27:05,832] ({qtp1582797472-104} RemoteInterpreter.java[call]:168) - Create RemoteInterpreter org.apache.zep
pelin.shell.ShellInterpreter
INFO [2019-08-21 13:27:05,928] ({qtp1582797472-104} NotebookServer.java[completion]:1413) - Fail to get completion
java.lang.RuntimeException: org.apache.thrift.TApplicationException: Internal error processing createInterpreter
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.callRemoteFunction(RemoteInterpreterProcess.java:139)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.internal_create(RemoteInterpreter.java:165)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.open(RemoteInterpreter.java:132)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.completion(RemoteInterpreter.java:349)
at org.apache.zeppelin.notebook.Paragraph.completion(Paragraph.java:270)
at org.apache.zeppelin.notebook.Note.completion(Note.java:729)
at org.apache.zeppelin.socket.NotebookServer.completion(NotebookServer.java:1411)
at org.apache.zeppelin.socket.NotebookServer.onMessage(NotebookServer.java:304)
at org.apache.zeppelin.socket.NotebookSocket.onWebSocketText(NotebookSocket.java:59)
at org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver.onTextMessage(JettyListenerEventDriver.java:189)
at org.eclipse.jetty.websocket.common.message.SimpleTextMessage.messageComplete(SimpleTextMessage.java:69)
at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.appendMessage(AbstractEventDriver.java:66)
at org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver.onTextFrame(JettyListenerEventDriver.java:158)
at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.incomingFrame(AbstractEventDriver.java:162)
at org.eclipse.jetty.websocket.common.WebSocketSession.incomingFrame(WebSocketSession.java:459)
at org.eclipse.jetty.websocket.common.extensions.AbstractExtension.nextIncomingFrame(AbstractExtension.java:182)
at org.eclipse.jetty.websocket.common.extensions.compress.PerMessageDeflateExtension.nextIncomingFrame(PerMessageDeflateExtension.java:105)
at org.eclipse.jetty.websocket.common.extensions.compress.CompressExtension.forwardIncoming(CompressExtension.java:142)
at org.eclipse.jetty.websocket.common.extensions.compress.PerMessageDeflateExtension.incomingFrame(PerMessageDeflateExtension.java:85)
at org.eclipse.jetty.websocket.common.extensions.ExtensionStack.incomingFrame(ExtensionStack.java:220)
at org.eclipse.jetty.websocket.common.Parser.notifyFrame(Parser.java:219)
at org.eclipse.jetty.websocket.common.Parser.parse(Parser.java:244)
at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.readParse(AbstractWebSocketConnection.java:559)
at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:390)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
at java.lang.Thread.run(Thread.java:745)
In the shell interpreter logs it translates to a ClassCastException
ERROR [2019-08-21 13:27:05,926] ({pool-1-thread-5} ProcessFunction.java[process]:41) - Internal error processing createInterpret
er
org.apache.thrift.TException: java.lang.ClassNotFoundException: org.apache.zeppelin.shell.ShellInterpreter
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer.createInterpreter(RemoteInterpreterServer.java:320)
at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Processor$createInterpreter.getResult(RemoteInterpret
erService.java:1799)
at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Processor$createInterpreter.getResult(RemoteInterpret
erService.java:1784)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: org.apache.zeppelin.shell.ShellInterpreter
at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:260)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer.createInterpreter(RemoteInterpreterServer.java:302)
... 8 more
After a little googling I found out that the issue is that zeppelin looks for a specific path when attempting to instantiate interpreters. However, when the shell interpreter is installed using the `install-interpreter.sh` script, it installs it to the `interpreter/shell` folder but the plugin system looks for the classes in the `interpreter/sh` folder.
A simple workaround for this issue is to rename the shell folder (`mv interpreter/shell interpreter/sh`) and restart the zeppelin daemon. But it would be good to fix the plugin system to look for the shell classes in the `interpreter/shell` directory instead of having manually rename the folder.