Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-25027

LegacyAccumulatorWrapper test fails in Scala 2.12

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Duplicate
    • 2.4.0
    • None
    • Build, Spark Core, Tests
    • None

    Description

      I've found at least one test failure in the 2.12; am running further to see if there are more.

      - LegacyAccumulatorWrapper with AccumulatorParam that has no equals/hashCode *** FAILED ***
      java.io.NotSerializableException: org.scalatest.Assertions$AssertionsHelper
      Serialization stack:
      - object not serializable (class: org.scalatest.Assertions$AssertionsHelper, value: org.scalatest.Assertions$AssertionsHelper@1b27bfe7)
      - field (class: org.scalatest.FunSuite, name: assertionsHelper, type: class org.scalatest.Assertions$AssertionsHelper)
      - object (class org.apache.spark.util.AccumulatorV2Suite, AccumulatorV2Suite)
      - field (class: org.apache.spark.util.AccumulatorV2Suite$$anon$1, name: $outer, type: class org.apache.spark.util.AccumulatorV2Suite)
      - object (class org.apache.spark.util.AccumulatorV2Suite$$anon$1, org.apache.spark.util.AccumulatorV2Suite$$anon$1@52d0064e)
      - field (class: org.apache.spark.util.LegacyAccumulatorWrapper, name: param, type: interface org.apache.spark.AccumulableParam)
      - object (class org.apache.spark.util.LegacyAccumulatorWrapper, LegacyAccumulatorWrapper(id: 358431, name: Some(test), value: org.apache.spark.util.AccumulatorV2Suite$MyData$1@7385bc7a))
      at org.apache.spark.serializer.SerializationDebugger$.improveException(SerializationDebugger.scala:41)
      at org.apache.spark.serializer.JavaSerializationStream.writeObject(JavaSerializer.scala:46)
      at org.apache.spark.serializer.JavaSerializerInstance.serialize(JavaSerializer.scala:100)
      at org.apache.spark.util.AccumulatorV2Suite.$anonfun$new$5(AccumulatorV2Suite.scala:182)
      at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
      at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
      at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
      at org.scalatest.Transformer.apply(Transformer.scala:22)
      at org.scalatest.Transformer.apply(Transformer.scala:20)
      at org.scalatest.FunSuiteLike$$anon$1.apply(FunSuiteLike.scala:186)
      ...
      

      Attachments

        Issue Links

          Activity

            People

              srowen Sean R. Owen
              srowen Sean R. Owen
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: