Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-27677

print out HiveMetaStore.audit to json format

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • None
    • None
    • Standalone Metastore
    • None

    Description

      This task aims to print a new[1] line of HiveMetaStore audit log in JSON format, similar as https://github.com/apache/hive/pull/1582 but extend to `cmd` details as well.

      1. existing audit log

      ```
      HiveMetaStore.audit: ugi=xxx ip=xx.xx.xx.xx cmd=source:xx.xx.xx.xx get_table : db=xxx tbl=xxx
      HiveMetaStore.audit: ugi=xxx ip=xx.xx.xx.xx cmd=source:xx.xx.xx.xx get_partition_with_auth : db=xx tbl=xx[xxx]
      ```

      1. The new audit log
        ```
        HiveMetaStore.audit: {ugi: "xxx", ip: "xx.xx.xx.xx", cmd={source: "xx.xx.xx.xx", api="get_table", params= {db: "xxx", tbl: "xxx"}

        }}
        HiveMetaStore.audit: {ugi: "xxx", ip: "xx.xx.xx.xx", cmd={source: "xx.xx.xx.xx", api="get_partition_with_auth", params=

        {db: "xxx", tbl: "xxx", key=["xxx"]}

        }}
        ```

      ----------------
      For some context, we're tracking the usage of the shared Hive Metastore Service. HiveMetaStore auditLog is the raw data we reply on, to understand the traffic on different dimensions, source(IP), API, database, table, etc. 

      Currently the audit log is in raw string without a standard format, especially for
      extraLogInfo, code point here, makes it harder to analyze.

      [1] should we print another line instead of replacing the existing one, to avoid a breaking-change?

       

      Attachments

        Activity

          People

            mingmxu Mingmin Xu
            mingmxu Mingmin Xu
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: