Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-24954

Fail fast on job submit if run a barrier stage with dynamic resource allocation enabled

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • 2.4.0
    • 2.4.0
    • Spark Core
    • None

    Description

      Since we explicitly listed "Support running barrier stage with dynamic resource allocation" a Non-Goal in the design doc, we shall fail fast on job submit if running a barrier stage with dynamic resource allocation enabled, to avoid some confusing behaviors (can refer to SPARK-24942 for some examples).

      Attachments

        Activity

          People

            jiangxb1987 Xingbo Jiang
            jiangxb1987 Xingbo Jiang
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: