Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-23489

Flaky Test: HiveExternalCatalogVersionsSuite

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.2.1, 2.3.0, 2.4.0
    • 2.2.2, 2.3.1, 2.4.0
    • SQL, Tests
    • None

    Description

      I saw this error in an unrelated PR. It seems to me a bad configuration in the Jenkins node where the tests are run.

      Error Message
      java.io.IOException: Cannot run program "./bin/spark-submit" (in directory "/tmp/test-spark/spark-2.2.1"): error=2, No such file or directory
      Stacktrace
      sbt.ForkMain$ForkError: java.io.IOException: Cannot run program "./bin/spark-submit" (in directory "/tmp/test-spark/spark-2.2.1"): error=2, No such file or directory
      	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
      	at org.apache.spark.sql.hive.SparkSubmitTestUtils$class.runSparkSubmit(SparkSubmitTestUtils.scala:73)
      	at org.apache.spark.sql.hive.HiveExternalCatalogVersionsSuite.runSparkSubmit(HiveExternalCatalogVersionsSuite.scala:43)
      	at org.apache.spark.sql.hive.HiveExternalCatalogVersionsSuite$$anonfun$beforeAll$1.apply(HiveExternalCatalogVersionsSuite.scala:176)
      	at org.apache.spark.sql.hive.HiveExternalCatalogVersionsSuite$$anonfun$beforeAll$1.apply(HiveExternalCatalogVersionsSuite.scala:161)
      	at scala.collection.immutable.List.foreach(List.scala:381)
      	at org.apache.spark.sql.hive.HiveExternalCatalogVersionsSuite.beforeAll(HiveExternalCatalogVersionsSuite.scala:161)
      	at org.scalatest.BeforeAndAfterAll$class.liftedTree1$1(BeforeAndAfterAll.scala:212)
      	at org.scalatest.BeforeAndAfterAll$class.run(BeforeAndAfterAll.scala:210)
      	at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:52)
      	at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:314)
      	at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:480)
      	at sbt.ForkMain$Run$2.call(ForkMain.java:296)
      	at sbt.ForkMain$Run$2.call(ForkMain.java:286)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      Caused by: sbt.ForkMain$ForkError: java.io.IOException: error=2, No such file or directory
      	at java.lang.UNIXProcess.forkAndExec(Native Method)
      	at java.lang.UNIXProcess.<init>(UNIXProcess.java:248)
      	at java.lang.ProcessImpl.start(ProcessImpl.java:134)
      	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
      	... 17 more
      

      This is the link: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87615/testReport/.

      MASTER BRANCH

      BRANCH 2.3

      NOTE: This failure frequently looks as `Test Result (no failures)`

      Attachments

        Issue Links

          Activity

            People

              dongjoon Dongjoon Hyun
              mgaido Marco Gaido
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: