Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-1620

livy.spark interpreter formatting breaks for scala with new line

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 0.6.0
    • None
    • livy-interpreter
    • HDP 2.5
      Apache Zeppelin

      Version 0.6.0.2.5.0.0-1245

    Description

      When using %livy.spark

      if you have new line and comments in multi-line statement it breaks. Same code using regular spark statement works correct.

      Error that you get:
      <console>:1: error: illegal start of definition
      .format("com.databricks.spark.csv")

      Example:
      %livy.spark

      val df = sqlContext.read
      .format("com.databricks.spark.csv")
      .option("header", "true") // Use first line of all files as header
      .option("inferSchema", "true") // Automatically infer data types
      .load("/tmp/airflightsdelays/") // Read all flights

      df.printSchema

      If i consolidate the code it works fine:
      %livy.spark

      val df = sqlContext.read.format("com.databricks.spark.csv").option("header", "true").option("inferSchema", "true").load("/tmp/airflightsdelays/") // Read all flights

      Attachments

        1. screenshot-1.png
          205 kB
          alex zeltov

        Issue Links

          Activity

            People

              Unassigned Unassigned
              azeltov alex zeltov
              Votes:
              12 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

                Created:
                Updated: