Details
-
New Feature
-
Status: Open
-
Major
-
Resolution: Unresolved
-
4.0.0
-
None
Description
We would like to develop a Java-based Kubernetes operator for Apache Spark. Following the operator pattern (https://kubernetes.io/docs/concepts/extend-kubernetes/operator/), Spark users may manage applications and related components seamlessly using native tools like kubectl. The primary goal is to simplify the Spark user experience on Kubernetes, minimizing the learning curve and operational complexities and therefore enable users to focus on the Spark application development.
Ideally, it would reside in a separate repository (like Spark docker or Spark connect golang) and be loosely connected to the Spark release cycle while supporting multiple Spark versions.
SPIP doc: https://docs.google.com/document/d/1f5mm9VpSKeWC72Y9IiKN2jbBn32rHxjWKUfLRaGEcLE
Dev email discussion : https://lists.apache.org/thread/wdy7jfhf7m8jy74p6s0npjfd15ym5rxz
Attachments
1.
|
Add License to Spark Operator | Resolved | Zhou JIANG | |
2.
|
Setup gradle as build tool for operator repository | Resolved | Zhou JIANG | |
3.
|
Setup Static Analysis for Operator | Resolved | Zhou JIANG | |
4.
|
Add Operator CI Task for Java Build and Test | Resolved | Zhou JIANG | |
5.
|
Add Java API Module for Spark Operator | Resolved | Zhou JIANG | |
6.
|
Update `build.gradle` to fix deprecation warnings | Resolved | Dongjoon Hyun | |
7.
|
Add Spark application submission worker for operator | Open | Unassigned |