Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13410

RunJar adds the content of the jar twice to the classpath

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.0.0-alpha1
    • util
    • None

    Description

      Today when you run a "hadoop jar" command, the jar is unzipped to a temporary location and gets added to the classloader.

      However, the original jar itself is still added to the classpath.

            List<URL> classPath = new ArrayList<>();
            classPath.add(new File(workDir + "/").toURI().toURL());
            classPath.add(file.toURI().toURL());
            classPath.add(new File(workDir, "classes/").toURI().toURL());
            File[] libs = new File(workDir, "lib").listFiles();
            if (libs != null) {
              for (File lib : libs) {
                classPath.add(lib.toURI().toURL());
              }
            }
      

      As a result, the contents of the jar are present in the classpath twice and are completely redundant. Although this does not necessarily cause correctness issues, some stricter code written to require a single presence of files may fail.

      I cannot think of a good reason why the jar should be added to the classpath if the unjarred content was added to it. I think we should remove the jar from the classpath.

      Attachments

        1. HADOOP-13410.001.patch
          2 kB
          Yuanbo Liu
        2. HADOOP-13410.002.patch
          1 kB
          Sangjin Lee

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            yuanbo Yuanbo Liu
            sjlee0 Sangjin Lee
            Votes:
            0 Vote for this issue
            Watchers:
            13 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment