[TOREE-420] AddDeps fails if transitive dependency includes org.scala-lang.modules - ASF JIRA

Agile Board

Attach files

Attach Screenshot

Voters

Stop watching

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: 0.2.0
Fix Version/s: 0.2.0
Component/s: Kernel
Labels:
None
Environment:
Spark 2.1.1, Scala 2.11.8, Built from source, commit id: c7b008506fa07538b42aa94b287a361a64f71ee6

Description

When using the AddDeps magic, if the dependency includes org.scala-lang.modules it fails with a reflection error, this kafka depdency for example:

%AddDeps org.apache.kafka kafka_2.11 0.10.2.0 --transitive --trace --verbose

{'buffers': [],
 'content': {'ename': 'scala.reflect.internal.FatalError',
             'evalue': 'object Predef does not have a member classOf',
             'traceback': ['scala.reflect.internal.Definitions$DefinitionsClass.scala$reflect$internal$Definitions$DefinitionsClass$$fatalMissingSymbol(Definitions.scala:1186)',
                           'scala.reflect.internal.Definitions$DefinitionsClass.getMember(Definitions.scala:1203)',
                           'scala.reflect.internal.Definitions$DefinitionsClass.getMemberMethod(Definitions.scala:1238)',
                           'scala.reflect.internal.Definitions$DefinitionsClass$RunDefinitions.Predef_classOf$lzycompute(Definitions.scala:1469)',
                           'scala.reflect.internal.Definitions$DefinitionsClass$RunDefinitions.Predef_classOf(Definitions.scala:1469)',
                           'scala.reflect.internal.Definitions$DefinitionsClass$RunDefinitions.isPredefClassOf(Definitions.scala:1459)',
                           'scala.tools.nsc.typechecker.Typers$Typer.typedIdent$2(Typers.scala:4885)',
                           'scala.tools.nsc.typechecker.Typers$Typer.typedIdentOrWildcard$1(Typers.scala:4908)',
                           'scala.tools.nsc.typechecker.Typers$Typer.typedInAnyMode$1(Typers.scala:5340)',
                           'scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5360)',
                           'scala.tools.nsc.typechecker.Typers$Typer.runTyper$1(Typers.scala:5396)',
                           'scala.tools.nsc.typechecker.Typers$Typer.scala$tools$nsc$typechecker$Typers$Typer$$typedInternal(Typers.scala:5423)',
                           'scala.tools.nsc.typechecker.Typers$Typer.body$2(Typers.scala:5370)',
                           'scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5374)',
                           'scala.tools.nsc.interpreter.ReplGlobal$$anon$1$$anon$2.typed(ReplGlobal.scala:36)',
                           'scala.tools.nsc.typechecker.Typers$Typer.typedQualifier(Typers.scala:5472)',
                           'scala.tools.nsc.typechecker.Typers$Typer.typedQualifier(Typers.scala:5480)',
                           'scala.tools.nsc.typechecker.Typers$Typer.typedPackageDef$1(Typers.scala:5012)',
                           'scala.tools.nsc.typechecker.Typers$Typer.typedMemberDef$1(Typers.scala:5312)',
                           'scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5359)',
                           'scala.tools.nsc.typechecker.Typers$Typer.runTyper$1(Typers.scala:5396)',
                           'scala.tools.nsc.typechecker.Typers$Typer.scala$tools$nsc$typechecker$Typers$Typer$$typedInternal(Typers.scala:5423)',
                           'scala.tools.nsc.typechecker.Typers$Typer.body$2(Typers.scala:5370)',
                           'scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5374)',
                           'scala.tools.nsc.interpreter.ReplGlobal$$anon$1$$anon$2.typed(ReplGlobal.scala:36)',
                           'scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5448)',
                           'scala.tools.nsc.typechecker.Analyzer$typerFactory$$anon$3.apply(Analyzer.scala:102)',
                           'scala.tools.nsc.Global$GlobalPhase$$anonfun$applyPhase$1.apply$mcV$sp(Global.scala:440)',
                           'scala.tools.nsc.Global$GlobalPhase.withCurrentUnit(Global.scala:431)',
                           'scala.tools.nsc.Global$GlobalPhase.applyPhase(Global.scala:440)',
                           'scala.tools.nsc.typechecker.Analyzer$typerFactory$$anon$3$$anonfun$run$1.apply(Analyzer.scala:94)',
                           'scala.tools.nsc.typechecker.Analyzer$typerFactory$$anon$3$$anonfun$run$1.apply(Analyzer.scala:93)',

However, the kafka 2.10 version does not fail:

%AddDeps org.apache.kafka kafka_2.10 0.10.2.0 --transitive --trace --verbose

After inspecting the pom files for each of the versions, the only difference was the dependency on

<dependency>
<groupId>org.scala-lang.modules</groupId>
<artifactId>scala-parser-combinators_2.11</artifactId>
<version>1.0.4</version>
<scope>compile</scope>

We should exclude the org.scala-lang.modules organization as well.
Including org.scala-lang.modules as a exclusion fixes this issue.