Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-44417

Spark Driver and executors are not transferring from standby master to Alive Master.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Critical
    • Resolution: Unresolved
    • 3.3.2
    • None
    • Deploy
    • None
    • RedHat linux 

      Zookeeper HA Cluster

    Description

      Issue with the Spark driver and executors transferring from Standby Master to Alive Master.

      Spark is deployed with Zookeeper as cluster manager. Ideally if standby node(server) goes down it should transfer applications running on standby node to alive node.
      Application transformations are killed in the Standby Master.
        # Just consider we have two Application drivers were running on the Standby Master server.

      1. If we forcefully shutdown, This should trigger the failover scenario and application should move to Alive node. But, The Standby Master node both the application drivers were not transferring to Active master instead those were killing.

      Check attachment for reference.
       
      Steps to reproduce

      • Deploy Spark HA cluster with Zookeeper
      • Deploy multiple application until we have drivers and executor assigned on standby node.
      • Now reboot the server
      • Check if all the applications were moving to Active Master. 
         
         

      Attachments

        1. Issue_steps.docx
          1017 kB
          Anand Changediya

        Activity

          People

            Unassigned Unassigned
            anandc Anand Changediya
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: