Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-28585

Improve WebUI DAG information: Add extra info to rdd from spark plan

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Won't Fix
    • Affects Version/s: 3.0.0
    • Fix Version/s: None
    • Component/s: Spark Core, Web UI
    • Labels:
      None

      Description

      The mainly improve that i want to achieve is to help developers to explore the DAG information in the Web UI in complex flows. 

      Sometimes is very dificult to know what part of your spark plan corresponds to the DAG you are looking.

       

      This is an initial improvement only in one simple spark plan type (UnionExec). 

      If you consider it a good idea, i want to extend it to other spark plans to improve the visualization iteratively.

       

      More info in the pull request 

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                planga82 Pablo Langa Blanco
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: