Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-28337

spark jars do not contain commons-jxpath jar, cause ClassNotFound exception

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Not A Problem
    • 3.0.0
    • None
    • Spark Submit
    • None

    Description

      When submit a spark application using XPathExpressionEngine, a ClassNotFound exception will be thrown, even though I shade then class of commons-jxpath:commons-jxpath into my jar.

      The main class of my jar is as follows:

      object StaticTestMain {
         def main(args: Array[String]): Unit = {

             println(s"yanzhi class loader info ${getClass.getClassLoader}")
             val engine = new XPathExpressionEngine() // exception happends in this line

              // some other codes.
         }
      }

      And configuration of shade plugin as the attached picture file "shade_in_pom.jpg"

      The exception stack trace is in the attached picture file "exception_stack_trace.jpg".

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            yanlin-Lynn Wang Yanlin
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment