Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-7717

Spark Standalone Web UI showing incorrect total memory, workers and cores

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 1.3.1
    • 1.5.0
    • Web UI
    • RedHat

    Description

      I launched a Spark master in standalone mode in one of my host and then launched 3 workers on three different hosts. The workers successfully connected to my master and the Web UI showed the correct details. Specifically, the Web UI correctly shows that the total memory and the total cores available for the cluster.

      However on one of the worker, I did a "kill -9 <worker process id>" and restarted the worker again. This time though, the master's Web UI shows incorrect total memory and number of cores. The total memory is shown to be 4*n, where "n" is the memory in each worker. Also the total workers is shown as 4 and the total number of cores shown is incorrect, it shows 4*c, where "c" is the number of cores on each worker.

      Attachments

        1. JIRA.PNG
          62 kB
          Swaranga Sarma

        Activity

          People

            zhichao-li zhichao-li
            swaranga Swaranga Sarma
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: