Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-39090

AddJar or AddFile call on a duplicate file should not result in failure

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Invalid
    • 3.1.0
    • None
    • Spark Submit
    • None

    Description

      Currently, when addFile or addJar is used to upload a file or a jar which was already uploaded, it should not result in IOException. This is because the spark-submit behavior is to ignore adding duplicate jars/files and logging appropriate warning message. The same should be done for addFile and addJar APIs in Livy.

      Attachments

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            praneetsharma Praneet Sharma
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment