Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-31980

Spark sequence() fails if start and end of range are identical dates

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.4.0, 2.4.1, 2.4.2, 2.4.3, 2.4.4, 2.4.5, 2.4.6
    • 2.4.7, 3.0.1, 3.1.0
    • SQL
    • None

    • Spark 2.4.4 standalone and on AWS EMR

    Description

       

      The following Spark SQL query throws an exception

      select sequence(cast("2011-03-01" as date), cast("2011-03-01" as date), interval 1 month)

      The error is:

       

       

      java.lang.ArrayIndexOutOfBoundsException: 1java.lang.ArrayIndexOutOfBoundsException: 1 at scala.runtime.ScalaRunTime$.array_update(ScalaRunTime.scala:92) at org.apache.spark.sql.catalyst.expressions.Sequence$TemporalSequenceImpl.eval(collectionOperations.scala:2681) at org.apache.spark.sql.catalyst.expressions.Sequence.eval(collectionOperations.scala:2514) at org.apache.spark.sql.catalyst.expressions.UnaryExpression.eval(Expression.scala:389)

       

      Attachments

        Activity

          People

            JinxinTang JinxinTang
            DaveDeCaprio Dave DeCaprio
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: