[HIVE-21563] Improve Table#getEmptyTable performance by disable registerAllFunctionsOnce - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 2.3.9, 3.1.3, 3.2.0, 4.0.0-alpha-1
Component/s: None
Labels:
- pull-request-available

Description

We do not need registerAllFunctionsOnce when Table#getEmptyTable. The stack trace:

  at org.apache.hadoop.hive.ql.exec.Registry.registerGenericUDF(Registry.java:177)
  at org.apache.hadoop.hive.ql.exec.Registry.registerGenericUDF(Registry.java:170)
  at org.apache.hadoop.hive.ql.exec.FunctionRegistry.<clinit>(FunctionRegistry.java:209)
  at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:247)
  at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:231)
  at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:388)
  at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:332)
  at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:312)
  at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:288)
  at org.apache.hadoop.hive.ql.session.SessionState.setAuthorizerV2Config(SessionState.java:913)
  at org.apache.hadoop.hive.ql.session.SessionState.setupAuth(SessionState.java:877)
  at org.apache.hadoop.hive.ql.session.SessionState.getAuthenticator(SessionState.java:1479)
  at org.apache.hadoop.hive.ql.session.SessionState.getUserFromAuthenticator(SessionState.java:1150)
  at org.apache.hadoop.hive.ql.metadata.Table.getEmptyTable(Table.java:180)

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

HIVE-21563.001.patch
02/Apr/19 04:59
2 kB
Yuming Wang

Issue Links

relates to

SPARK-35321 Spark 3.x can't talk to HMS 1.2.x and lower due to get_all_functions Thrift API missing

Resolved

links to

GitHub Pull Request #2251

Activity

People

Assignee:: Yuming Wang

Reporter:: Yuming Wang

Votes:: 0 Vote for this issue

Watchers:: 5 Start watching this issue

Dates

Created:: 02/Apr/19 04:44

Updated:: 21/Oct/22 06:56

Resolved:: 07/May/21 05:32

Time Tracking

Estimated:

Not Specified

Remaining:

Logged:

10m