Details
-
New Feature
-
Status: Open
-
Major
-
Resolution: Unresolved
-
1.18.0
-
None
-
None
Description
We are using Flink (1.18) version for our Flink cluster. The job manager has been deployed in "Application mode" and we are looking for a process to restore multiple jobs (using their respective savepoint directories) when the job manager is started. Currently, we have the option to restore only one job while running "standalone-job.sh" using the --fromSavepoint and --allowNonRestoredState. However, we need a way to trigger multiple job executions via Java client (from its respective savepoint location) on Jobmanager startup.
Note: We are not using a Kubernetes native deployment, but we are using k8s standalone mode of deployment.
Additional Query: If there is a process to restore multiple jobs from its respective savepoints on "Application mode" of deployment, is the same supported on Session mode of deployment or not?
Expected process:
- Before starting with the Flink/application image upgrade, trigger the savepoints for all the current running jobs.
- Once the savepoints process completed for all jobs, will trigger the scale down of job manager and task manager instances.
- Update the image version on the k8s deployment with the update application image.
- After image version is updated, scale up the job manager and task manager.
- We need a process to restore the previously running jobs from the savepoint dir and start all the jobs.