Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-28585

Improve WebUI DAG information: Add extra info to rdd from spark plan

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Won't Fix
    • 3.0.0
    • None
    • Spark Core, Web UI
    • None

    Description

      The mainly improve that i want to achieve is to help developers to explore the DAG information in the Web UI in complex flows. 

      Sometimes is very dificult to know what part of your spark plan corresponds to the DAG you are looking.

       

      This is an initial improvement only in one simple spark plan type (UnionExec). 

      If you consider it a good idea, i want to extend it to other spark plans to improve the visualization iteratively.

       

      More info in the pull request 

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              planga82 Pablo Langa Blanco
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: