[HIVE-28064] Add cause to ParseException for diagnosability purposes - ASF JIRA

Log work

Agile Board

Rank to Top

Rank to Bottom

Bulk Copy Attachments

Bulk Move Attachments

Voters

Watch issue

Watchers

Create sub-task

Convert to sub-task

Move

Link

Clone

Labels

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 4.0.0
Component/s: Parser
Labels:
- pull-request-available

Description

The ParseException contains high level information about problems encountered during parsing but currently the stacktrace is pretty shallow.

select * from author where  true > fname in ('Alex','Rob')

Currently the query above will fail with the above stacktrace:

org.apache.hadoop.hive.ql.parse.ParseException: line 4:41 cannot recognize input near 'in' '(' ''Alex'' in expression specification
        at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:125)
        at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:97)
        at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:89)
        at org.apache.hadoop.hive.ql.Compiler.parse(Compiler.java:172)
        at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:105)
        at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:519)
        at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:471)
        at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:436)
        at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:430)
        at org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:121)
        at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:227)
        at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:257)
        at org.apache.hadoop.hive.cli.CliDriver.processCmd1(CliDriver.java:201)
        at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:127)
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:425)
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:356)

The end-user gets a hint about what the error might be but the developer has no way to tell how far we went into parsing the given statement and which grammar rule failed to pass.

In some cases we are catching the RecognitionException which shows exactly at which point the parser failed to advance so we could pass it as a cause inside the ParseException. With this change the stacktrace would be similar to the one below:

org.apache.hadoop.hive.ql.parse.ParseException: line 4:41 cannot recognize input near 'in' '(' ''Alex'' in expression specification
        at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:125)
        at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:97)
        at org.apache.hadoop.hive.ql.parse.ParseUtils.parse(ParseUtils.java:89)
        at org.apache.hadoop.hive.ql.Compiler.parse(Compiler.java:172)
        at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:105)
        at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:519)
        at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:471)
        at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:436)
        at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:430)
        at org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:121)
        at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:227)
        at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:257)
        at org.apache.hadoop.hive.cli.CliDriver.processCmd1(CliDriver.java:201)
        at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:127)
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:425)
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:356)
Caused by: NoViableAltException(173@[()* loopback of 739:5: (equal= precedenceEqualOperator p= precedenceSimilarExpression -> ^( $equal $p) |dist= precedenceDistinctOperator p= precedenceSimilarExpression -> ^( KW_NOT["not"] ^( EQUAL_NS["<=>"] $p) ) )*])
        at org.antlr.runtime.DFA.noViableAlt(DFA.java:158)
        at org.antlr.runtime.DFA.predict(DFA.java:116)
        at org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceEqualExpression(HiveParser_IdentifiersParser.java:11812)
        at org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceUnarySuffixExpression(HiveParser_IdentifiersParser.java:12370)
        at org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceNotExpression(HiveParser_IdentifiersParser.java:12560)
        at org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceAndExpression(HiveParser_IdentifiersParser.java:12669)
        at org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.precedenceOrExpression(HiveParser_IdentifiersParser.java:12810)
        at org.apache.hadoop.hive.ql.parse.HiveParser_IdentifiersParser.expression(HiveParser_IdentifiersParser.java:8566)
        at org.apache.hadoop.hive.ql.parse.HiveParser.expression(HiveParser.java:45010)
        at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.searchCondition(HiveParser_FromClauseParser.java:6622)
        at org.apache.hadoop.hive.ql.parse.HiveParser_FromClauseParser.whereClause(HiveParser_FromClauseParser.java:6540)
        at org.apache.hadoop.hive.ql.parse.HiveParser.whereClause(HiveParser.java:45058)
        at org.apache.hadoop.hive.ql.parse.HiveParser.atomSelectStatement(HiveParser.java:38576)
        at org.apache.hadoop.hive.ql.parse.HiveParser.selectStatement(HiveParser.java:38849)
        at org.apache.hadoop.hive.ql.parse.HiveParser.regularBody(HiveParser.java:38442)
        at org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpressionBody(HiveParser.java:37704)
        at org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpression(HiveParser.java:37592)
        at org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:2763)
        at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:1640)
        at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:123)
        ... 59 more

End-users still get the same high-level error message and developers can see exactly at which point the parser crashed as well as which rule might be worth investigating.