[SPARK-24877] Ignore the task completion event from a zombie barrier task - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Open
Priority: Major
Resolution: Unresolved
Affects Version/s: 3.1.0
Fix Version/s: None
Component/s: Spark Core
Labels:
None

Epic Link:
Support Barrier Execution Mode

Description

Currently we abort the barrier stage if a zombie barrier task can't get killed to prevent data correctness issue. We can improve the behavior to let zombie barrier task continue running but not able to interact with other barrier tasks (maybe from different stage attempt) and ignore the task completion event from a zombie barrier task.

Attachments

Activity

People

Assignee:: Unassigned

Reporter:: Xingbo Jiang

Votes:: 1 Vote for this issue

Watchers:: 3 Start watching this issue

Dates

Created:: 20/Jul/18 16:17

Updated:: 16/Mar/20 22:50