Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-25648

HiveMetaHook not work well in HiveMetaStoreClient when commitCreateTable table failed!

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.1.2
    • None
    • API, Hooks, Standalone Metastore
    • None

    Description

      // source code in  HiveMetaStoreClient.java -- createTable func
        public void createTable(Table tbl, EnvironmentContext envContext) throws AlreadyExistsException,
            InvalidObjectException, MetaException, NoSuchObjectException, TException {
          if (!tbl.isSetCatName()) {
            tbl.setCatName(getDefaultCatalog(conf));
          }
          HiveMetaHook hook = getHook(tbl);
          if (hook != null) {
            hook.preCreateTable(tbl);
          }
          boolean success = false;
          try {
            // Subclasses can override this step (for example, for temporary tables)
            create_table_with_environment_context(tbl, envContext);  *//create metadata record*
            if (hook != null) {
              hook.commitCreateTable(tbl);  *//create table in external catalog*
            }
            success = true;
          }
          finally {
            if (!success && (hook != null)) {
              try {
      * // roll back from external catalog but without roll back from hive meta*
                hook.rollbackCreateTable(tbl); 
              } catch (Exception e){
                LOG.error("Create rollback failed with", e);
              }
            }
          }
        }
      

      Accoriding to the source code above, when implementing hivemetastore's HiveMetaHook to create external catalog tables(may be hbase),firstly create meta records to the database such as pg, then call the commitCreateTable function to create table in hbase. Here comes the question: What if exception thrown when creating the real table in hbase, because meta data has been created so it is not in sync between Hive's metastore and hbase.
      I think it is necessary to rollback metadata from hivemetastore when failed to create table in external catalog by calling commitCreateTable, so that we can keep external catalog in sync with Hive's metastore.
      Please let me know if my idea is correct or I had an misunderstanding on how to use the HiveMetaHook mechanism correctly!

      Attachments

        Activity

          People

            Unassigned Unassigned
            Stephen_Liu Jun Liu
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated: