Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-30272

Remove usage of Guava that breaks in Guava 27

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0
    • 3.0.0
    • Spark Core, SQL
    • None

    Description

      Background:
      https://issues.apache.org/jira/browse/SPARK-29250
      https://github.com/apache/spark/pull/25932

      Hadoop 3.2.1 will update Guava from 11 to 27. There are a number of methods that changed between those releases, typically just a rename, but, means one set of code can't work with both, while we want to work with Hadoop 2.x and 3.x. Among them:

      • Objects.toStringHelper was moved to MoreObjects; we can just use the Commons Lang3 equivalent
      • Objects.hashCode etc were renamed; use java.util.Objects equivalents
      • MoreExecutors.sameThreadExecutor() became directExecutor(); for same-thread execution we can use a dummy implementation of ExecutorService / Executor
      • TypeToken.isAssignableFrom become isSupertypeOf; work around with reflection

      There is probably more to the Guava issue than just this change, but it will make Spark itself work with more versions and reduce our exposure to Guava along the way anyway.

      Attachments

        Issue Links

          Activity

            People

              srowen Sean R. Owen
              srowen Sean R. Owen
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: