Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-4323

Kotlin support for Spark interpreter

Agile BoardAttach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      This issue aims to add Kotlin language support in Apache Zeppelin, adding a standalone interpreter as well as an interpreter for Spark group.

      Summary:

      • Features:
        • Kotlin 1.3.50 support in standalone %kotlin interpreter
        • Kotlin with Spark 2.4.x support in Spark interpreter, available under %spark.kotlin (Kotlin uses Java Spark API)
        • ZeppelinContext support in Spark interpreter that allows visualisations, object sharing, etc.
        • A bound KotlinContext that provides access to bound and user-defined variables and functions. Example:
          val x = listOf(1, 2, 3);
          kc.showVars()

          will print

          kc: KotlinContext! = org.apache.zeppelin.kotlin.repl.KotlinRepl$KotlinContext@2ae76226
          x: List<Int> = [1, 2, 3]
      • Implementation details:
        • Kotlin interpreter uses Kotlin REPL tools to interpret code.
        • Variable and function binding is done via implicit receiver object passed to REPL, so user code is executed inside Kotlin's with block, making receiver's fields and methods visible in the scope.
        • REPL generated classes can be written to spark.repl.classes.outputDir, making them available in Spark jobs.
      • Licensing
        • All Kotlin libraries have Apache 2.0 license.
      • Documentation
        • Kotlin documentation is added to ./docs/interpreter, and Spark documentation is updated
      • Testing
        • Kotlin interpreter has unit tests in its module, and Kotlin Spark interpreter tests are added.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ileasile Ilya Muradyan
            dkaznacheev Dmitry
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - Not Specified
              Not Specified
              Remaining:
              Remaining Estimate - 0h
              0h
              Logged:
              Time Spent - 13h 10m
              13h 10m

              Slack

                Issue deployment