Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-27947

Enhance redactOptions to accept any Map type

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0
    • 3.0.0
    • SQL
    • None

    Description

      In ParsedStatement.productIterator, `case mapArg: Map[_, _]` may match any Map type, thus causing `asInstanceOf[Map[String, String]]` to throw ClassCastException.

      The following test reproduces the issue:

      case class TestStatement(p: Map[String, Int]) extends ParsedStatement {
       override def output: Seq[Attribute] = Nil
       override def children: Seq[LogicalPlan] = Nil
      }
      
      TestStatement(Map("abc" -> 1)).toString

      Changing the code to `case mapArg: Map[String, String]` will not work due to type erasure. As a matter of fact, compiler gives this warning:

      Warning:(41, 18) non-variable type argument String in type pattern scala.collection.immutable.Map[String,String] (the underlying of Map[String,String]) is unchecked since it is eliminated by erasure
      case mapArg: Map[String, String] =>

       

      Attachments

        Issue Links

          Activity

            People

              jzhuge John Zhuge
              jzhuge John Zhuge
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: