Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-15058 [Umbrella] Current Flaky Tests
  3. HIVE-15289

Flaky test: TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver (setup)

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      In recent PreCommit builds, TestSparkCliDriver has failed during setup with errors like the following:

      From https://builds.apache.org/job/PreCommit-HIVE-Build/2292/testReport/:

      Failed during createSources processLine with code=3
      ...
      Job failed with java.io.IOException: Failed to create local dir in /tmp/blockmgr-be4539eb-7896-4903-89c9-7ae1c48faa24/01.
      	at org.apache.spark.storage.DiskBlockManager.getFile(DiskBlockManager.scala:70)
      	at org.apache.spark.storage.DiskStore.contains(DiskStore.scala:124)
      	at org.apache.spark.storage.BlockManager.org$apache$spark$storage$BlockManager$$getCurrentBlockStatus(BlockManager.scala:379)
      	at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:959)
      	at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:910)
      	at org.apache.spark.storage.BlockManager.doPut(BlockManager.scala:866)
      	at org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:910)
      	at org.apache.spark.storage.BlockManager.putIterator(BlockManager.scala:700)
      	at org.apache.spark.storage.BlockManager.putSingle(BlockManager.scala:1213)
      	at org.apache.spark.broadcast.TorrentBroadcast.writeBlocks(TorrentBroadcast.scala:103)
      	at org.apache.spark.broadcast.TorrentBroadcast.<init>(TorrentBroadcast.scala:86)
      	at org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast(TorrentBroadcastFactory.scala:34)
      	at org.apache.spark.broadcast.BroadcastManager.newBroadcast(BroadcastManager.scala:56)
      	at org.apache.spark.SparkContext.broadcast(SparkContext.scala:1370)
      	at org.apache.spark.rdd.HadoopRDD.<init>(HadoopRDD.scala:125)
      	at org.apache.spark.SparkContext$$anonfun$hadoopRDD$1.apply(SparkContext.scala:965)
      	at org.apache.spark.SparkContext$$anonfun$hadoopRDD$1.apply(SparkContext.scala:961)
      	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
      	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
      	at org.apache.spark.SparkContext.withScope(SparkContext.scala:682)
      	at org.apache.spark.SparkContext.hadoopRDD(SparkContext.scala:961)
      	at org.apache.spark.api.java.JavaSparkContext.hadoopRDD(JavaSparkContext.scala:412)
      	at org.apache.hadoop.hive.ql.exec.spark.SparkPlanGenerator.generateMapInput(SparkPlanGenerator.java:205)
      	at org.apache.hadoop.hive.ql.exec.spark.SparkPlanGenerator.generateParentTran(SparkPlanGenerator.java:145)
      	at org.apache.hadoop.hive.ql.exec.spark.SparkPlanGenerator.generate(SparkPlanGenerator.java:117)
      	at org.apache.hadoop.hive.ql.exec.spark.RemoteHiveSparkClient$JobStatusJob.call(RemoteHiveSparkClient.java:339)
      	at org.apache.hive.spark.client.RemoteDriver$JobWrapper.call(RemoteDriver.java:358)
      	at org.apache.hive.spark.client.RemoteDriver$JobWrapper.call(RemoteDriver.java:323)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      

      From https://builds.apache.org/job/PreCommit-HIVE-Build/2291/testReport/:

      Failed during createSources processLine with code=1
      ...
      Failed to monitor Job[ 11] with exception 'org.apache.hadoop.hive.ql.metadata.HiveException(java.util.concurrent.TimeoutException)'
      

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              erwaman Anthony Hsu
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: