Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-20926

Exposure to Guava libraries by directly accessing tableRelationCache in SessionCatalog caused failures

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.2.0
    • 2.2.0, 2.3.0
    • Spark Core
    • None

    Description

      Because of shading that we did for guava libraries, we see test failures whenever those components directly access tableRelationCache in SessionCatalog.
      This can happen in any component that shaded guava library. Failures looks like this:

      java.lang.NoSuchMethodError: org.apache.spark.sql.catalyst.catalog.SessionCatalog.tableRelationCache()Lcom/google/common/cache/Cache;
      01:25:14   at org.apache.spark.sql.hive.test.TestHiveSparkSession.reset(TestHive.scala:492)
      01:25:14   at org.apache.spark.sql.hive.test.TestHiveContext.reset(TestHive.scala:138)
      01:25:14   at org.apache.spark.sql.hive.test.TestHiveSingleton$class.afterAll(TestHiveSingleton.scala:32)
      01:25:14   at org.apache.spark.sql.hive.StatisticsSuite.afterAll(StatisticsSuite.scala:34)
      01:25:14   at org.scalatest.BeforeAndAfterAll$class.afterAll(BeforeAndAfterAll.scala:213)
      01:25:14   at org.apache.spark.SparkFunSuite.afterAll(SparkFunSuite.scala:31)
      01:25:14   at org.scalatest.BeforeAndAfterAll$$anonfun$run$1.apply(BeforeAndAfterAll.scala:280)
      01:25:14   at org.scalatest.BeforeAndAfterAll$$anonfun$run$1.apply(BeforeAndAfterAll.scala:278)
      01:25:14   at org.scalatest.CompositeStatus.whenCompleted(Status.scala:377)
      01:25:14   at org.scalatest.BeforeAndAfterAll$class.run(BeforeAndAfterAll.scala:278)
      

      Attachments

        Activity

          People

            rezasafi Reza Safi
            rezasafi Reza Safi
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: