Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-8983

Spark uber jar job server: query exceptions from master

Details

    • Improvement
    • Status: Open
    • P3
    • Resolution: Unresolved
    • None
    • None
    • runner-spark

    Description

      As far as I know, the Spark REST API does not return exceptions from the cluster after a jar is actually run. While these exceptions can be viewed in Spark's web UI, ideally they would also be visible in Beam's output.

      To do this, we will need to find a REST endpoint that does return those exceptions, and then map the submissionId to its corresponding job id.

      Attachments

        Activity

          People

            Unassigned Unassigned
            ibzib Kyle Weaver
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: