Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-25075 Build and test Spark against Scala 2.13
  3. SPARK-34507

Spark artefacts built against Scala 2.13 incorrectly depend on Scala 2.12

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.2.0
    • 2.4.8, 3.0.3, 3.1.2, 3.2.0
    • Build
    • None

    Description

      Snapshots of Spark 3.2 built against Scala 2.13 are available at https://repository.apache.org/content/repositories/snapshots/org/apache/spark/, but they seem to depend on Scala 2.12. Specifically if I look at https://repository.apache.org/content/repositories/snapshots/org/apache/spark/spark-parent_2.13/3.2.0-SNAPSHOT/spark-parent_2.13-3.2.0-20210223.010629-29.pom I see:

      <scala.version>2.12.10</scala.version>
      <scala.binary.version>2.13</scala.binary.version

      It looks like https://github.com/apache/spark/blob/8f994cbb4a18558c2e81516ef1e339d9c8fa0d41/dev/change-scala-version.sh#L65 needs to be updated to also change the `scala.version` and not just the `scala.binary.version`.

      Attachments

        Activity

          People

            srowen Sean R. Owen
            smarter Guillaume Martres
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: