Description
Currently, Spark SQL can throw Java exceptions from the timestampadd()/date_add()/dateadd() functions, for instance:
spark-sql> select timestampadd(YEAR, 1000000, timestamp'2022-03-09 01:02:03'); 22/03/09 14:47:15 ERROR SparkSQLDriver: Failed in [select timestampadd(YEAR, 1000000, timestamp'2022-03-09 01:02:03')] java.lang.ArithmeticException: long overflow at java.lang.Math.multiplyExact(Math.java:892) ~[?:1.8.0_292] at org.apache.spark.sql.catalyst.util.DateTimeUtils$.instantToMicros(DateTimeUtils.scala:505) ~[spark-catalyst_2.12-3.3.0-SNAPSHOT.jar:3.3.0-SNAPSHOT] at org.apache.spark.sql.catalyst.util.DateTimeUtils$.timestampAddMonths(DateTimeUtils.scala:724) ~[spark-catalyst_2.12-3.3.0-SNAPSHOT.jar:3.3.0-SNAPSHOT] at org.apache.spark.sql.catalyst.util.DateTimeUtils$.timestampAdd(DateTimeUtils.scala:1197) ~[spark-catalyst_2.12-3.3.0-SNAPSHOT.jar:3.3.0-SNAPSHOT]
That might confuse non-Scala/Java users. Need to wrap such kind of exception by Spark's exception using an error class.
Attachments
Issue Links
- is a clone of
-
SPARK-37858 Throw Spark exceptions from AES functions
-
- Resolved
-
- links to
