Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-33005 Kubernetes GA Preparation
  3. SPARK-34334

ExecutorPodsAllocator fails to identify some excess requests during downscaling

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 3.0.0, 3.0.1, 3.1.0, 3.1.1, 3.1.2, 3.2.0
    • 3.1.1, 3.2.0
    • Kubernetes
    • None

    Description

      During downscaling there are two kinds of POD requests which can lead to POD deletion as it is identified as excess requests by the dynamic allocation:

      • timed out newly created POD requests (which even haven't reached the Pod Pending state, yet)
      • timed out pending pod requests

      The current implementation fails to delete a timed out pending pod requests when there are not enough timed out newly created POD requests to delete but there are some non-timed out ones.

      Attachments

        Activity

          People

            attilapiros Attila Zsolt Piros
            attilapiros Attila Zsolt Piros
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: