[HIVE-24073] Execution exception in sort-merge semijoin - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: None
Component/s: Operators
Labels:
- pull-request-available

Description

Working on ~~HIVE-24041~~, we trigger an additional SJ conversion that leads to this exception at execution time:

Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: Attempting to overwrite nextKeyWritables[1]
	at org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:1063)
	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:685)
	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:707)
	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:707)
	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:707)
	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:707)
	at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.close(MapRecordProcessor.java:462)
	... 16 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: Attempting to overwrite nextKeyWritables[1]
	at org.apache.hadoop.hive.ql.exec.GroupByOperator.flush(GroupByOperator.java:1037)
	at org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:1060)
	... 22 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Attempting to overwrite nextKeyWritables[1]
	at org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.processKey(CommonMergeJoinOperator.java:564)
	at org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.process(CommonMergeJoinOperator.java:243)
	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:887)
	at org.apache.hadoop.hive.ql.exec.TezDummyStoreOperator.process(TezDummyStoreOperator.java:49)
	at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:887)
	at org.apache.hadoop.hive.ql.exec.GroupByOperator.forward(GroupByOperator.java:1003)
	at org.apache.hadoop.hive.ql.exec.GroupByOperator.flush(GroupByOperator.java:1020)
	... 23 more

To reproduce, just set hive.auto.convert.sortmerge.join to true in the last query in auto_sortmerge_join_10.q after ~~HIVE-24041~~ has been merged.

Attachments

Issue Links

relates to

HIVE-24041 Extend semijoin conversion rules

Closed

links to

GitHub Pull Request #1476

Activity

People

Assignee:: mahesh kumar behera

Reporter:: Jesús Camacho Rodríguez

Votes:: 0 Vote for this issue

Watchers:: 1 Start watching this issue

Dates

Created:: 25/Aug/20 21:55

Updated:: 27/Feb/24 22:13

Resolved:: 08/Jun/21 02:07

Time Tracking

Estimated:

Not Specified

Remaining:

Logged:

1h 40m