Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-20955

Calcite Rule HiveExpandDistinctAggregatesRule seems throwing IndexOutOfBoundsException

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 4.0.0
    • None
    • None
    • None

    Description

       

      Adde the following query to Druid test  ql/src/test/queries/clientpositive/druidmini_expressions.q

      select count(distinct `__time`, cint) from (select * from druid_table_alltypesorc) as src;
      
      

      leads to error {code} 2018-11-21T07:36:39,449 ERROR [main] QTestUtil: Client execution failed with error code = 40000 running "{code}

      with exception stack 

      
      2018-11-21T07:36:39,443 ERROR [ecd48683-0286-4cb4-b0ad-e150fab51038 main] parse.CalcitePlanner: CBO failed, skipping CBO.
      java.lang.IndexOutOfBoundsException: index (1) must be less than size (1)
       at com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:310) ~[guava-19.0.jar:?]
       at com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:293) ~[guava-19.0.jar:?]
       at com.google.common.collect.SingletonImmutableList.get(SingletonImmutableList.java:41) ~[guava-19.0.jar:?]
       at org.apache.calcite.rel.metadata.RelMdColumnOrigins.getColumnOrigins(RelMdColumnOrigins.java:77) ~[calcite-core-1.17.0.jar:1.17.0]
       at GeneratedMetadataHandler_ColumnOrigin.getColumnOrigins_$(Unknown Source) ~[?:?]
       at GeneratedMetadataHandler_ColumnOrigin.getColumnOrigins(Unknown Source) ~[?:?]
       at org.apache.calcite.rel.metadata.RelMetadataQuery.getColumnOrigins(RelMetadataQuery.java:345) ~[calcite-core-1.17.0.jar:1.17.0]
       at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveExpandDistinctAggregatesRule.onMatch(HiveExpandDistinctAggregatesRule.java:168) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
       at org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:315) ~[calcite-core-1.17.0.jar:1.17.0]
       at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:556) ~[calcite-core-1.17.0.jar:1.17.0]
       at org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:415) ~[calcite-core-1.17.0.jar:1.17.0]
       at org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:280) ~[calcite-core-1.17.0.jar:1.17.0]
       at org.apache.calcite.plan.hep.HepInstruction$RuleCollection.execute(HepInstruction.java:74) ~[calcite-core-1.17.0.jar:1.17.0]
       at org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:211) ~[calcite-core-1.17.0.jar:1.17.0]
       at org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:198) ~[calcite-core-1.17.0.jar:1.17.0]
       at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.hepPlan(CalcitePlanner.java:2363) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
       at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.hepPlan(CalcitePlanner.java:2314) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
       at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.applyPreJoinOrderingTransforms(CalcitePlanner.java:2031) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
       at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1780) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
       at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1680) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
       at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:118) ~[calcite-core-1.17.0.jar:1.17.0]
       at org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:1043) ~[calcite-core-1.17.0.jar:1.17.0]
       at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:154) ~[calcite-core-1.17.0.jar:1.17.0]
       at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:111) ~[calcite-core-1.17.0.jar:1.17.0]
       at org.apache.hadoop.hive.ql.parse.CalcitePlanner.logicalPlan(CalcitePlanner.java:1439) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
       at org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:478) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
       at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12296) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
       at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:358) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
       at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:288) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
       at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:670) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
       at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1893) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
       at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1840) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
       at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1835) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
       at org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
       at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:219) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
       at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:242) [hive-cli-4.0.0-SNAPSHOT.jar:?]
       at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:189) [hive-cli-4.0.0-SNAPSHOT.jar:?]
       at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:408) [hive-cli-4.0.0-SNAPSHOT.jar:?]
       at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:341) [hive-cli-4.0.0-SNAPSHOT.jar:?]
      
      

      Attachments

        1. HIVE-20955.4.patch
          7 kB
          Slim Bouguerra
        2. HIVE-20955.3.patch
          6 kB
          Vineet Garg
        3. HIVE-20955.2.patch
          6 kB
          Vineet Garg
        4. HIVE-20955.1.patch
          1 kB
          Vineet Garg

        Issue Links

          Activity

            People

              vgarg Vineet Garg
              bslim Slim Bouguerra
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: