Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-3050

Parse Error if the spark job has the spark.executor.extraClassPath confguration.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 4.1.0
    • None
    • client, core, workflow
    • None

    Description

      1. create a spark job workflow in Oozie
      2. add --conf spark.executor.extraClassPath=/XXX/XXX in job confuration
      3. submit the job
      Results:
      the Oozie workflow failed and the error log is as follows. it has an addtion error "--conf".
      --conf
      --conf
      spark.executor.extraClassPath=aaa:$PWD/*
      --conf
      spark.driver.extraClassPath=$PWD/*

      Error: Spark config without '=': --conf
      Run with --help for usage help or --verbose for debug output
      Intercepting System.exit(1)
      Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SparkMain], exit code [1]

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              Tang Yan Tang Yan
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: