Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-27272

Enable blacklisting of node/executor on fetch failures by default

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Incomplete
    • 2.3.0
    • None
    • Spark Core

    Description

      SPARK-20898 added a new configuration to blacklist a node/executor on fetch
      failures. This config was deemed risky at the time and was disabled by default
      until more data is collected.

      This commit aims to enable that feature by default as we have seen couple of
      instances where that feature was found to be useful. The failures occur because of issues with node manager and external shuffle service on that node.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              ankur.gupta Ankur Gupta
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: