Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-15829

spark master webpage links to application UI broke when running in cluster mode

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Incomplete
    • 1.6.1
    • None
    • EC2
    • AWS ec2 cluster

    Description

      Hi
      I created a cluster using the spark-1.6.1-bin-hadoop2.6/ec2/spark-ec2

      I use the stand alone cluster manager. I have a streaming app running in cluster mode. I notice the master webpage links to the application UI page are incorrect

      It does not look like jira will let my upload images. I'll try and describe the web pages and the bug

      My master is running on
      http://ec2-54-215-230-73.us-west-1.compute.amazonaws.com:8080/

      It has a section marked "applications". If I click on one of the running application ids I am taken to a page showing "Executor Summary". This page has a link to teh 'application detail UI' the url is http://ec2-54-215-230-73.us-west-1.compute.amazonaws.com:4041/

      Notice it things the application UI is running on the cluster master.

      It is actually running on the same machine as the driver on port 4041. I was able to reverse engine the url by noticing the private ip address is part of the worker id . For example worker-20160322041632-172.31.23.201-34909

      next I went on the aws ec2 console to find the public DNS name for this machine
      http://ec2-54-193-104-169.us-west-1.compute.amazonaws.com:4041/streaming/

      Kind regards

      Andy

      Attachments

        Activity

          People

            Unassigned Unassigned
            aedwip Andrew Davidson
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: