Uploaded image for project: 'Livy'
  1. Livy
  2. LIVY-712

EMR 5.23/5.27 - Livy does not recognise that Spark job failed

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Workaround
    • 0.5.0, 0.6.0
    • None
    • API
    • AWS EMR 5.23/5.27, Scala

    Description

      We've upgraded from AWS EMR 5.13 -> 5.23 (Livy 0.4.0 -> 0.5.0, Spark 2.3.0 -> 2.4.0) and an issue appears that when there is an exception thrown during Spark job execution, Spark shuts down as if there was no problem and job appears as Completed in EMR. So we're not notified when system crashes. The same problem appears in EMR 5.27 (Livy 0.6.0, Spark 2.4.4).

      Is it something with Spark? Or a known issue with Livy?

      In Livy logs I see that spark-submit exists with error code 1

      05:34:59 WARN BatchSession$: spark-submit exited with code 1

       And then Livy API states that batch state is

      "state": "success"

      How can it be made work again?

      Attachments

        Activity

          People

            Unassigned Unassigned
            sankotm Michal Sankot
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: