Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
0.8.0
-
None
-
None
-
HDP: 3.0.1
zeppelin: 0.8.0
Spark: 2.3.1
Description
I am testing use of scala case class in a zeppelin instance provided by an Horontoworks Data Platform stack.
The bug is reproducible with following instructions:
Copy/paste in zeppelin instructions from https://zeppelin.apache.org/docs/0.5.5-incubating/tutorial/tutorial.html. (full screenshot attached)
In a first cell, instructions to create the RDD and then the temporary table.
After execution, I get "console>:37: error: value toDF is not a member of org.apache.spark.rdd.RDD[Bank] bank.toDF().registerTempTable("bank")
Second cell is the first sql query in the tutorial.
Execution fails with:
java.lang.ArrayIndexOutOfBoundsException: 5
at $line541920741114.$read$$iw$$iw$$anonfun$3.apply(<console>:33)
at $line541920741114.$read$$iw$$iw$$anonfun$3.apply(<console>:29)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)
...
The exact basic tutorial code (i.e. access to amazone and building a local in memory rdd) seems to work in remote zeppelin (i.e. sql provides results) but I still get "value toDF is not a member of org.apache.spark.rdd.RDD[Bank] ".
Testing the basic tutorial code in a local 0.8.0 zeppelin works without any issue.
Using code for building an in memory