Details

    Description

      DataConnectorProviderFactory uses HashMap to cache data connector instances, and there is no way to invalidate the cache unless you restart the MetaStore.  
      What is more serious is that if you drop or alter the dataconnector, the cache will not change, and you maybe use a invalid dataconnector next time.

       

      I think we can improve the dataconnector cache from the two aspects:

      • Use Caffeine with a maxmumsize e.g. 100  to cache data connector instead of HashMap, and set a expire time after the last accessing. And we also should close the underlying  datasource connection using Caffeine RemovalListener.
      • After executing Drop or Alter DDL on a dataConnector, we should update cache to clean the dataConnector to avoid using the invalid dataConnector next time.

      Attachments

        Issue Links

          Activity

            People

              zhangbutao Butao Zhang
              zhangbutao Butao Zhang
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: