Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-26941

Make SetProcessor configurable to ignore some set variables

    XMLWordPrintableJSON

Details

    Description

      In certain environments after upgrades we need to restrict users from changing some Hive configurations during runtime (for example "mapreduce.job.queuename" or "hive.execution.engine").
      The "hive.security.authorization.sqlstd.confwhitelist" could be used for this, however:

      • it is complex to modify that sometimes (for example to exclude a config which is otherwise allowed with a wildcard)
      • when a user script tries to set a parameter not in the "hive.security.authorization.sqlstd.confwhitelist" then the whole script just fails with "Error: Error while processing statement: Cannot modify xxxx at runtime. It is not in list of params that are allowed to be modified at runtime". This would require all the user scripts and jobs to be modified (to remove that "set" command), that can be a huge effort.

      With a new configuration item in hive-site.xml cluster operators can configure HiveServer2 to ignore the "set" command requests - essentially making those settings "final" on HiveServer2 level. Trying to change these "final" settings would not fail the scripts - just ignore their request.

      In this jira:

      • add a new config "hive.conf.ignored.variable.list"
      • accepts strings, comma separated list of variables
      • the config is empty by default, it can be set in hive-site.xml only
      • adding the "hive.conf.ignored.variable.list" to the restricted list ("hive.conf.restricted.list") internally - so it cannot be modified during runtime
      • adding tests for the changes

      Attachments

        Activity

          People

            mszurap Miklos Szurap
            mszurap Miklos Szurap
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: