Spark Kubernetes Operator

We would like to develop a Java-based Kubernetes operator for Apache Spark. Following the operator pattern (https://kubernetes.io/docs/concepts/extend-kubernetes/operator/), Spark users may manage applications and related components seamlessly using native tools like kubectl. The primary goal is to simplify the Spark user experience on Kubernetes, minimizing the learning curve and operational complexities and therefore enable users to focus on the Spark application development.

Ideally, it would reside in a separate repository (like Spark docker or Spark connect golang) and be loosely connected to the Spark release cycle while supporting multiple Spark versions.

SPIP doc: https://docs.google.com/document/d/1f5mm9VpSKeWC72Y9IiKN2jbBn32rHxjWKUfLRaGEcLE

Dev email discussion : https://lists.apache.org/thread/wdy7jfhf7m8jy74p6s0npjfd15ym5rxz