Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-26614

Speculation kill might cause job failure

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.1.0
    • 2.2.2, 2.3.1, 2.4.0
    • Scheduler, Spark Core
    • None

    Description

      This issue is similar to SPARK-26612

      Some odd exceptions might be thrown in speculation kill, however, currently spark does not handle this case and will report failure to Driver. This exception will be counting towards MAX_TASK_FAILURES, and might result in the job failure.

       

      I think we can check state of task to tell if we report failure or killed.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              liupengcheng liupengcheng
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: