Details
-
Sub-task
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
Description
DataConnectorProviderFactory uses HashMap to cache data connector instances, and there is no way to invalidate the cache unless you restart the MetaStore.
What is more serious is that if you drop or alter the dataconnector, the cache will not change, and you maybe use a invalid dataconnector next time.
I think we can improve the dataconnector cache from the two aspects:
- Use Caffeine with a maxmumsize e.g. 100 to cache data connector instead of HashMap, and set a expire time after the last accessing. And we also should close the underlying datasource connection using Caffeine RemovalListener.
- After executing Drop or Alter DDL on a dataConnector, we should update cache to clean the dataConnector to avoid using the invalid dataConnector next time.
Attachments
Issue Links
- links to