Details
-
Improvement
-
Status: Resolved
-
P2
-
Resolution: Fixed
-
None
-
None
Description
BEAM-4449 replaces our use of Project and Filter with a compound operation Calc via these rules:
- FilterToCalcRule.INSTANCE,
- ProjectToCalcRule.INSTANCE,
Which then allows these optimizations:
- FilterCalcMergeRule.INSTANCE,
- ProjectCalcMergeRule.INSTANCE,
- CalcRemoveRule.INSTANCE,
- CalcMergeRule.INSTANCE,
This always hits assertion errors in the VolcanoPlanner when CalcRemoveRule is invoked, because:
- LogicalCalc.create always results in a composite collation trait
- VolcanoPlanner asserts that there are no composite traits
- There's no simplify() in between the two (I'm not sure there can be)
The stack trace (Calcite 1.16.0) is this:
java.lang.AssertionError
at org.apache.calcite.plan.volcano.VolcanoPlanner.changeTraits(VolcanoPlanner.java:546)
at org.apache.calcite.plan.RelOptRule.convert(RelOptRule.java:540)
at org.apache.calcite.rel.rules.CalcRemoveRule.onMatch(CalcRemoveRule.java:65)
at org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:212)
at org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:650)
at org.apache.calcite.tools.Programs$RuleSetProgram.run(Programs.java:368)
at org.apache.calcite.prepare.PlannerImpl.transform(PlannerImpl.java:336)
at org.apache.beam.sdk.extensions.sql.impl.BeamQueryPlanner.convertToBeamRel(BeamQueryPlanner.java:135)
We can actually make progress by disabling CalcRemoveRule in a couple of places. Filing this ticket to track turning it back on since there is no real reason we shouldn't have it.
Attachments
Issue Links
- is related to
-
BEAM-6810 [SQL] Plan optimizer eliminates aliasing in trivial projections
- Resolved
- links to