Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-30821

Executor pods with multiple containers will not be rescheduled unless all containers fail

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.1.0
    • 3.0.2, 3.1.0
    • Kubernetes, Spark Core
    • None

    Description

      Since the restart policy of launched pods is Never, additional handling is required for pods that may have sidecar containers. The executor should be considered failed if any containers have terminated and have a non-zero exit code, but Spark currently only checks the pod phase. The pod phase will remain "running" as long as any pods are still running. Kubernetes sidecar support in 1.18/1.19 does not address this situation, as sidecar containers are excluded from pod phase calculation.

      Attachments

        Activity

          People

            huskysun Shiqi Sun
            hogeland Kevin Hogeland
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: