Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-31324

StreamingQuery stop() timeout exception should include the stream ID

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Trivial
    • Resolution: Fixed
    • 3.1.0
    • 3.0.0
    • Structured Streaming
    • None

    Description

      https://github.com/apache/spark/pull/26771/ added a conf to set a timeout when stop()ing streams; if the stream does not terminate in this time, an exception is thrown. Having hit this issue once in production, it would be nice to have the stream ID in the error message.

      Attachments

        Issue Links

          relates to

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-30143 StreamingQuery.stop() should not block indefinitely

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #28095

          Activity

            People

              Unassigned Unassigned
              mukulmurthy Mukul Murthy
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: