Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-28762

Read JAR main class if JAR is not located in local file system

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 3.0.0
    • Fix Version/s: 3.0.0
    • Component/s: Deploy, Spark Core, Spark Submit
    • Labels:
      None

      Description

      Currently, spark-submit doesn't attempt to read the main class from a Spark app JAR file if the scheme of the primary resource URI is not file. In other words, if the JAR is not in the local file system, it will barf.

      It would be useful to have this feature if I deploy my Spark app JARs in S3 or HDFS.

      If it makes sense to maintainers, I can take a stab at this - I think I know which files to look at.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                igozali Ivan Gozali
                Reporter:
                igozali Ivan Gozali
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: