Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-10444

Remove duplication in Mesos schedulers

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Incomplete
    • 1.5.0
    • None
    • Mesos

    Description

      Currently coarse-grained and fine-grained Mesos schedulers don't share much code, and that leads to inconsistencies. For instance:

      • only coarse-grained mode respects spark.cores.max, see SPARK-9873
      • only coarse-grained mode blacklists slaves that fail repeatedly, but that seams like generally useful
      • constraints and memory checking are done on both sides (code is shared though)
      • framework re-registration (master election) is only done for cluster-mode deployment

      We should find a better design that groups together common concerns and generally improves the code.

      Attachments

        Activity

          People

            Unassigned Unassigned
            dragos Dragos Dascalita Haut
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: