Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-22733

After disable operation log property in hive, still HS2 saving the operation log

    XMLWordPrintableJSON

Details

    Description

      There are few issues in this area.
      1. If logging is disabled using hive.server2.logging.operation.enabled, then operation logs for the queries should not be generated. But the registerLoggingContext method in LogUtils, registers the logging context  even if the operation log is disabled. This causes the logs to be added by logger. The registration of query context should be done only if operation logging is enabled.

       public static void registerLoggingContext(Configuration conf) {
      -    MDC.put(SESSIONID_LOG_KEY, HiveConf.getVar(conf, HiveConf.ConfVars.HIVESESSIONID));
      -    MDC.put(QUERYID_LOG_KEY, HiveConf.getVar(conf, HiveConf.ConfVars.HIVEQUERYID));
           if (HiveConf.getBoolVar(conf, HiveConf.ConfVars.HIVE_SERVER2_LOGGING_OPERATION_ENABLED)) {
      +      MDC.put(SESSIONID_LOG_KEY, HiveConf.getVar(conf, HiveConf.ConfVars.HIVESESSIONID));
      +      MDC.put(QUERYID_LOG_KEY, HiveConf.getVar(conf, HiveConf.ConfVars.HIVEQUERYID));
             MDC.put(OPERATIONLOG_LEVEL_KEY, HiveConf.getVar(conf, HiveConf.ConfVars.HIVE_SERVER2_LOGGING_OPERATION_LEVEL));

       

      2. In case of failed query, we close the operations and that deletes the logging context (appender and route) from logger for that query. But if any log is added after that, the query logs are getting added and new operation log file is getting generated for the query. This looks like issue with MCD clear. MCD clear is not removing the keys from the map. If remove is used instead of clear, its working fine.

      Attachments

        1. HIVE-22733.02.patch
          5 kB
          mahesh kumar behera
        2. HIVE-22733.01.patch
          5 kB
          mahesh kumar behera

        Issue Links

          Activity

            People

              maheshk114 mahesh kumar behera
              maheshk114 mahesh kumar behera
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 20m
                  1h 20m