Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
3.0.0
-
None
Description
We need to rename the following Jenkins jobs first.
spark-master-test-sbt-hadoop-2.7 -> spark-master-test-sbt-hadoop-2.7-hive-1.2
spark-master-test-sbt-hadoop-3.2 -> spark-master-test-sbt-hadoop-3.2-hive-2.3
spark-master-test-maven-hadoop-2.7 -> spark-master-test-maven-hadoop-2.7-hive-1.2
spark-master-test-maven-hadoop-3.2 -> spark-master-test-maven-hadoop-3.2-hive-2.3
Also, we need to add `-Phive-1.2` for the existing `hadoop-2.7` jobs.
-Phive \ + -Phive-1.2 \
And, we need to add `-Phive-2.3` for the existing `hadoop-3.2` jobs.
-Phive \ + -Phive-2.3 \
Now now, I added the above `-Phive-1.2` and `-Phive-2.3` to the Jenkins manually. (This should be added to SCM of AmpLab Jenkins.)
After SPARK-29981, we need to create two new jobs.
- spark-master-test-sbt-hadoop-2.7-hive-2.3
- spark-master-test-maven-hadoop-2.7-hive-2.3
This is for preparation for Apache Spark 3.0.0.
We may drop all `*-hive-1.2` jobs at Apache Spark 3.1.0.