Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-30034 Use Apache Hive 2.3 dependency by default
  3. SPARK-29988

Adjust Jenkins jobs for `hive-1.2/2.3` combination

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0
    • 3.0.0
    • Project Infra
    • None

    Description

      We need to rename the following Jenkins jobs first.

      spark-master-test-sbt-hadoop-2.7 -> spark-master-test-sbt-hadoop-2.7-hive-1.2
      spark-master-test-sbt-hadoop-3.2 -> spark-master-test-sbt-hadoop-3.2-hive-2.3
      spark-master-test-maven-hadoop-2.7 -> spark-master-test-maven-hadoop-2.7-hive-1.2
      spark-master-test-maven-hadoop-3.2 -> spark-master-test-maven-hadoop-3.2-hive-2.3

      Also, we need to add `-Phive-1.2` for the existing `hadoop-2.7` jobs.

              -Phive \
      +        -Phive-1.2 \
      

      And, we need to add `-Phive-2.3` for the existing `hadoop-3.2` jobs.

              -Phive \
      +        -Phive-2.3 \
      

      Now now, I added the above `-Phive-1.2` and `-Phive-2.3` to the Jenkins manually. (This should be added to SCM of AmpLab Jenkins.)

      After SPARK-29981, we need to create two new jobs.

      • spark-master-test-sbt-hadoop-2.7-hive-2.3
      • spark-master-test-maven-hadoop-2.7-hive-2.3

      This is for preparation for Apache Spark 3.0.0.
      We may drop all `*-hive-1.2` jobs at Apache Spark 3.1.0.

      Attachments

        Activity

          People

            shaneknapp Shane Knapp
            dongjoon Dongjoon Hyun
            Votes:
            0 Vote for this issue
            Watchers:
            11 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: