Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-46384

Structured Streaming UI doesn't display graph correctly

    XMLWordPrintableJSON

Details

    Description

      The Streaming UI is broken currently at spark master. Running a simple query:

      ```

      q = spark.readStream.format("rate").load().writeStream.format("memory").queryName("test_wei").start()

      ```

      Would make the spark UI shows empty graph for "operation duration":

      Here is the error:

       

      I verified the same query runs fine on spark 3.5, as in the following graph.

       

      This should be a problem from the library updates, this could be a potential source of error: https://github.com/apache/spark/pull/42879

       

       

      Attachments

        Issue Links

          Activity

            People

              yao Kent Yao
              WweiL Wei Liu
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: