Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-4323

Kotlin support for Spark interpreter

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      This issue aims to add Kotlin language support in Apache Zeppelin, adding a standalone interpreter as well as an interpreter for Spark group.

      Summary:

      • Features:
        • Kotlin 1.3.50 support in standalone %kotlin interpreter
        • Kotlin with Spark 2.4.x support in Spark interpreter, available under %spark.kotlin (Kotlin uses Java Spark API)
        • ZeppelinContext support in Spark interpreter that allows visualisations, object sharing, etc.
        • A bound KotlinContext that provides access to bound and user-defined variables and functions. Example:
          val x = listOf(1, 2, 3);
          kc.showVars()

          will print

          kc: KotlinContext! = org.apache.zeppelin.kotlin.repl.KotlinRepl$KotlinContext@2ae76226
          x: List<Int> = [1, 2, 3]
      • Implementation details:
        • Kotlin interpreter uses Kotlin REPL tools to interpret code.
        • Variable and function binding is done via implicit receiver object passed to REPL, so user code is executed inside Kotlin's with block, making receiver's fields and methods visible in the scope.
        • REPL generated classes can be written to spark.repl.classes.outputDir, making them available in Spark jobs.
      • Licensing
        • All Kotlin libraries have Apache 2.0 license.
      • Documentation
        • Kotlin documentation is added to ./docs/interpreter, and Spark documentation is updated
      • Testing
        • Kotlin interpreter has unit tests in its module, and Kotlin Spark interpreter tests are added.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ileasile Ilya Muradyan
            dkaznacheev Dmitry
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 13h 10m
                13h 10m

                Slack

                  Issue deployment