Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-33944

Apache Flink: Process to restore more than one job on job manager startup from the respective savepoints

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 1.18.0
    • None
    • None

    Description

       
      We are using Flink (1.18) version for our Flink cluster. The job manager has been deployed in "Application mode" and we are looking for a process to restore multiple jobs (using their respective savepoint directories) when the job manager is started. Currently, we have the option to restore only one job while running "standalone-job.sh" using the --fromSavepoint and --allowNonRestoredState. However, we need a way to trigger multiple job executions via Java client (from its respective savepoint location) on Jobmanager startup.

      Note: We are not using a Kubernetes native deployment, but we are using k8s standalone mode of deployment.

      Additional Query: If there is a process to restore multiple jobs from its respective savepoints on "Application mode" of deployment, is the same supported on Session mode of deployment or not?

      Expected process:

      1. Before starting with the Flink/application image upgrade, trigger the savepoints for all the current running jobs.
      2. Once the savepoints process completed for all jobs, will trigger the scale down of job manager and task manager instances.
      3. Update the image version on the k8s deployment with the update application image.
      4. After image version is updated, scale up the job manager and task manager.
      5. We need a process to restore the previously running jobs from the savepoint dir and start all the jobs.

      Attachments

        Activity

          People

            Unassigned Unassigned
            vrangana@in.ibm.com Vijay
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: