Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-4323

Kotlin support for Spark interpreter

    XMLWordPrintableJSON

Details

    Description

      This issue aims to add Kotlin language support in Apache Zeppelin, adding a standalone interpreter as well as an interpreter for Spark group.

      Summary:

      • Features:
        • Kotlin 1.3.50 support in standalone %kotlin interpreter
        • Kotlin with Spark 2.4.x support in Spark interpreter, available under %spark.kotlin (Kotlin uses Java Spark API)
        • ZeppelinContext support in Spark interpreter that allows visualisations, object sharing, etc.
        • A bound KotlinContext that provides access to bound and user-defined variables and functions. Example:
          val x = listOf(1, 2, 3);
          kc.showVars()

          will print

          kc: KotlinContext! = org.apache.zeppelin.kotlin.repl.KotlinRepl$KotlinContext@2ae76226
          x: List<Int> = [1, 2, 3]
      • Implementation details:
        • Kotlin interpreter uses Kotlin REPL tools to interpret code.
        • Variable and function binding is done via implicit receiver object passed to REPL, so user code is executed inside Kotlin's with block, making receiver's fields and methods visible in the scope.
        • REPL generated classes can be written to spark.repl.classes.outputDir, making them available in Spark jobs.
      • Licensing
        • All Kotlin libraries have Apache 2.0 license.
      • Documentation
        • Kotlin documentation is added to ./docs/interpreter, and Spark documentation is updated
      • Testing
        • Kotlin interpreter has unit tests in its module, and Kotlin Spark interpreter tests are added.

      Attachments

        Issue Links

          Activity

            People

              ileasile Ilya Muradyan
              dkaznacheev Dmitry
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 13h 10m
                  13h 10m