Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-1385

Fail to create SparkSession when connecting to remote Spark

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Duplicate
    • 0.6.1
    • None
    • zeppelin-interpreter
    • None

    Description

      When connecting to remote Spark cluster, fails to execute job with NPE when trying to create Spark Session because outputDir has not been assigned yet.

      DEBUG [2016-08-28 11:42:12,733] (

      {pool-2-thread-2} Logging.scala[logDebug]:58) - Adding shutdown hook
      DEBUG [2016-08-28 11:42:14,735] ({pool-2-thread-2}

      Interpreter.java[getProperty]:179) - key: master, value: spark://192.168.1.102:7077
      INFO [2016-08-28 11:42:14,735] (

      {pool-2-thread-2} SparkInterpreter.java[createSparkSession]:299) - ------ Create new SparkContext spark://192.168.1.102:7077 -------
      DEBUG [2016-08-28 11:42:14,735] ({pool-2-thread-2}

      Interpreter.java[getProperty]:179) - key: spark.app.name, value: Zeppelin
      DEBUG [2016-08-28 11:42:14,735] (

      {pool-1-thread-5}

      Interpreter.java[getProperty]:179) - key: master, value: spark://192.168.1.102:7077
      ERROR [2016-08-28 11:42:14,736] (

      {pool-2-thread-2}

      Job.java[run]:182) - Job failed
      java.lang.NullPointerException
      at org.apache.zeppelin.spark.SparkInterpreter.createSparkSession(SparkInterpreter.java:307)
      at org.apache.zeppelin.spark.SparkInterpreter.getSparkSession(SparkInterpreter.java:218)
      at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:748)
      at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
      at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93)
      at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:385)
      at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
      at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:745)

      Attachments

        Issue Links

          Activity

            People

              lresende Luciano Resende
              lresende Luciano Resende
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: