Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
Impala 2.8.0
Description
Impala folds constants in a few specific places. For example, for predicates assigned to Kudu scan nodes, or for predicates used as partition filters. This constant folding may result in queries not being executable if the evaluation result is NaN or infinity (and possibly other special values).
The underlying problem is that our NumeritLiteral has no good way of representing NaN or infinity.
Repro:
explain select * from functional.alltypes where year < (cast(1.1000000238418579 as double) / cast(0 as double)); ERROR: NumberFormatException: Infinite or NaN
Stack:
I1110 13:04:23.117324 16414 jni-util.cc:169] java.lang.NumberFormatException: Infinite or NaN at java.math.BigDecimal.<init>(BigDecimal.java:808) at org.apache.impala.analysis.LiteralExpr.create(LiteralExpr.java:202) at org.apache.impala.analysis.Expr.foldConstantChildren(Expr.java:1219) at org.apache.impala.planner.HdfsPartitionPruner.canEvalUsingPartitionMd(HdfsPartitionPruner.java:182) at org.apache.impala.planner.HdfsPartitionPruner.prunePartitions(HdfsPartitionPruner.java:123) at org.apache.impala.planner.SingleNodePlanner.createHdfsScanPlan(SingleNodePlanner.java:1199) at org.apache.impala.planner.SingleNodePlanner.createScanNode(SingleNodePlanner.java:1259) at org.apache.impala.planner.SingleNodePlanner.createTableRefNode(SingleNodePlanner.java:1482) at org.apache.impala.planner.SingleNodePlanner.createTableRefsPlan(SingleNodePlanner.java:757) at org.apache.impala.planner.SingleNodePlanner.createSelectPlan(SingleNodePlanner.java:597) at org.apache.impala.planner.SingleNodePlanner.createQueryPlan(SingleNodePlanner.java:248) at org.apache.impala.planner.SingleNodePlanner.createSingleNodePlan(SingleNodePlanner.java:145) at org.apache.impala.planner.Planner.createPlan(Planner.java:91) at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:987) at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1067) at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:155)
The easiest solution is to consider the constant folding a failure for special evaluation results like NaN/infinity and fall back to executing the original expression.
As a result, the index-based partition pruning may not kick in or predicates may not be pushed to Kudu. These cases seem exotic enough for this behavior to be acceptable in most cases.
Alternatively, we could work on proper representation of NaN and infinity in our NumericLiteral.