Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-33005 Kubernetes GA Preparation
  3. SPARK-34166

Fix flaky test in DecommissionWorkerSuite

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 3.1.0, 3.1.1, 3.2.0
    • 3.1.1
    • Spark Core, Tests
    • None

    Description

      The test `decommission workers ensure that shuffle output is regenerated even with shuffle service` assumes it has two executor and both of two tasks can execute concurrently.

      The two tasks will execute serially if there only one executor. The result is test is unexpceted. E.g. 

      ```

      [info] 5 did not equal 4 Expected 4 tasks but got List(0:0:0:0-SUCCESS, 0:0:1:0-FAILED, 0:0:1:1-SUCCESS, 0:1:0:0-SUCCESS, 1:0:0:0-SUCCESS) (DecommissionWorkerSuite.scala:190)

      ```

      Attachments

        Activity

          People

            ulysses XiDuo You
            ulysses XiDuo You
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: