Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-6955

Do not let Yarn Shuffle Server retry its server port.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 1.2.0
    • 1.4.0
    • Shuffle, Spark Core, YARN
    • None

    Description

      It's better to let the NodeManager get down rather than take a port retry when `spark.shuffle.service.port` has been conflicted during starting the Spark Yarn Shuffle Server, because the retry mechanism will make the inconsistency of shuffle port and also make client fail to find the port.

      Attachments

        Activity

          People

            adav Aaron Davidson
            carlmartin Zhaowei Huang
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: