Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-27939

Many UNION ALL throws SemanticException when trying to remove partition predicates: fail to find child from parent

    XMLWordPrintableJSON

Details

    Description

      I found that the ticket for HIVE-26779 alone does not resolve when using many UNION ALL. When we create a DDL with ddl.sql and execute a query with query.sql, we get a SemanticException similar to HIVE-26779.

      23/12/07 18:02:01 ERROR ql.Driver: FAILED: SemanticException Exception when trying to remove partition predicates: fail to find child from parent
org.apache.hadoop.hive.ql.parse.SemanticException: Exception when trying to remove partition predicates: fail to find child from parent
        at org.apache.hadoop.hive.ql.exec.Operator.removeChildAndAdoptItsChildren(Operator.java:809)
        at org.apache.hadoop.hive.ql.parse.GenTezUtils.removeUnionOperators(GenTezUtils.java:472)
        at org.apache.hadoop.hive.ql.parse.TezCompiler.generateTaskTree(TezCompiler.java:691)
        at org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:301)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.compilePlan(SemanticAnalyzer.java:13054)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:13272)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12628)
        at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
        at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:224)
        at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:107)
        at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:519)
        at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:471)
        at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:436)
        at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:430)
        at org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:121)
        at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:227)
        at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:257)
        at org.apache.hadoop.hive.cli.CliDriver.processCmd1(CliDriver.java:201)
        at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:127)
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:425)
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:356)
        at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:509)
        at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:525)
        at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:843)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:807)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:721)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:236)

      Attachments

        1. query.sql
          22 kB
          Ryu Kobayashi
        2. ddl.sql
          2 kB
          Ryu Kobayashi

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. HIVE-27858 OOM happens when selecting many columns and JOIN.

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #4950

          Web Link GitHub Pull Request #4992

          Activity

            People

              ryu_kobayashi Ryu Kobayashi
              ryu_kobayashi Ryu Kobayashi
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: