[HIVE-24113] NPE in GenericUDFToUnixTimeStamp - ASF JIRA

Details

Type: Bug
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: 3.1.2
Fix Version/s: 4.0.0-alpha-1
Component/s: Hive
Labels:
None

Description

Following query will trigger the getPartitionsByExpr call at HMS, HMS will try to evaluate the filter based on the PartitionExpressionForMetastore proxy, this proxy uses the QL packages to evaluate the filter and call GenericUDFToUnixTimeStamp.

select * from table_name where hour between from_unixtime(unix_timestamp('2020090120', 'yyyyMMddHH') - 1*60*60, 'yyyyMMddHH') and from_unixtime(unix_timestamp('2020090122', 'yyyyMMddHH') + 2*60*60, 'yyyyMMddHH');

I think SessionState in the code path will always be NULL thats why it hit the NPE.

java.lang.NullPointerException: null
        at org.apache.hadoop.hive.ql.udf.generic.GenericUDFToUnixTimeStamp.initializeInput(GenericUDFToUnixTimeStamp.java:126) ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
        at org.apache.hadoop.hive.ql.udf.generic.GenericUDFToUnixTimeStamp.initialize(GenericUDFToUnixTimeStamp.java:75) ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
        at org.apache.hadoop.hive.ql.udf.generic.GenericUDF.initializeAndFoldConstants(GenericUDF.java:148) ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
        at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:146) ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
        at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:140) ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
        at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:140) ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
        at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:140) ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
        at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:140) ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
        at org.apache.hadoop.hive.ql.optimizer.ppr.PartExprEvalUtils.prepareExpr(PartExprEvalUtils.java:119) ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
        at org.apache.hadoop.hive.ql.optimizer.ppr.PartitionPruner.prunePartitionNames(PartitionPruner.java:551) ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
        at org.apache.hadoop.hive.ql.optimizer.ppr.PartitionExpressionForMetastore.filterPartitionsByExpr(PartitionExpressionForMetastore.java:82) ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
        at org.apache.hadoop.hive.metastore.ObjectStore.getPartitionNamesPrunedByExprNoTxn(ObjectStore.java:3527) ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
        at org.apache.hadoop.hive.metastore.ObjectStore.access$1400(ObjectStore.java:252) ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
        at org.apache.hadoop.hive.metastore.ObjectStore$10.getJdoResult(ObjectStore.java:3493) ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
        at org.apache.hadoop.hive.metastore.ObjectStore$10.getJdoResult(ObjectStore.java:3464) ~[hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
        at org.apache.hadoop.hive.metastore.ObjectStore$GetHelper.run(ObjectStore.java:3764) [hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
        at org.apache.hadoop.hive.metastore.ObjectStore.getPartitionsByExprInternal(ObjectStore.java:3499) [hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
        at org.apache.hadoop.hive.metastore.ObjectStore.getPartitionsByExpr(ObjectStore.java:3452) [hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_112]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_112]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_112]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_112]
        at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97) [hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
        at com.sun.proxy.$Proxy28.getPartitionsByExpr(Unknown Source) [?:?]
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_partitions_by_expr(HiveMetaStore.java:6637) [hive-exec-3.1.0.3.1.5.65-1.jar:3.1.0.3.1.5.65-1]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_112]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.ja

Attachments

Issue Links

Is contained by

HIVE-26751 Bug Fixes and Improvements for 3.2.0 release

Open

NPE in GenericUDFToUnixTimeStamp

Details

Description

Attachments

Issue Links

Activity

People

Dates