Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-27360 Standalone cluster mode support for GPU-aware scheduling
  3. SPARK-27371

Standalone master receives resource info from worker and allocate driver/executor properly

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0
    • 3.0.0
    • Spark Core
    • None

    Description

      As an admin, I can let Spark Standalone worker automatically discover GPUs installed on worker nodes. So I don't need to manually configure them.

      Attachments

        Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. SPARK-31921 Wrong warning of "WARN Master: App app-xxx requires more resource than any of Workers could have."

          • Major - Major loss of function.
          • Resolved
          is blocked by

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. SPARK-27365 Spark Jenkins supports testing GPU-aware scheduling features

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #25047

          Activity

            People

              Ngone51 wuyi
              mengxr Xiangrui Meng
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: