Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-31329

Modify executor monitor to allow for moving shuffle blocks

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • 3.1.0
    • None
    • Kubernetes, Spark Core
    • None

    Description

      To enable Spark-20629 we need to revisit code that assumes shuffle blocks don't move. Currently, the executor monitor assumes that shuffle blocks are immovable. We should modify this code.

      Attachments

        Issue Links

          Blocked

          Sub-task - The sub-task of the issue SPARK-20629 Copy shuffle data when nodes are being shut down

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              holden Holden Karau
              holden Holden Karau
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: