Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-21856

SQLStdHiveAuthorizationValidator checkPrivileges should not check DFS when authorizing select calls on table

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 4.0.0
    • Fix Version/s: None
    • Component/s: HiveServer2
    • Labels:
      None

      Description

      I encountered strange problem in the master branch build. With the following hive-site.xml authorization related properties. 
      <property>
      <name>hive.server2.enable.doAs</name>
      <value>false</value>
      </property>
      <property>
      <name>hive.security.authorization.enabled</name>
      <value>true</value>
      </property>
      <property>
      <name>hive.security.authorization.manager</name>
      <value>org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAuthorizerFactory</value>
      </property>
       
      I am seeing this backtrace. 
      2019-06-10 10:08:53,964 INFO sqlstd.SQLAuthorizationUtils: Checking fs privileges of user hive for HdfsLocatedFileStatus{path=hdfs://localhost:9000/tmp/hive/resultscache/results-1c53b375-a40a-44d2-9340-f08285cf80e0/8c0b4faf-0768-4534-a00f-6df4e2147f64; isDirectory=true; modification_time=1560186530192; access_time=0; owner=sam.an; group=supergroup; permission=rwxr-xr-x; isSymlink=false; hasAcl=false; isEncrypted=false; isErasureCoded=false} recursively

      FAILED: HiveAuthzPluginException Error getting permissions for /tmp/hive/resultscache/results-1c53b375-a40a-44d2-9340-f08285cf80e0/8c0b4faf-0768-4534-a00f-6df4e2147f64: null

      2019-06-10 10:08:54,650 ERROR ql.Driver: FAILED: HiveAuthzPluginException Error getting permissions for /tmp/hive/resultscache/results-1c53b375-a40a-44d2-9340-f08285cf80e0/8c0b4faf-0768-4534-a00f-6df4e2147f64: null

      org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthzPluginException: Error getting permissions for /tmp/hive/resultscache/results-1c53b375-a40a-44d2-9340-f08285cf80e0/8c0b4faf-0768-4534-a00f-6df4e2147f64: null

      at org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLAuthorizationUtils.getPrivilegesFromFS(SQLAuthorizationUtils.java:422)

      at org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAuthorizationValidator.checkPrivileges(SQLStdHiveAuthorizationValidator.java:123)

      at org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAuthorizationValidator.checkPrivileges(SQLStdHiveAuthorizationValidator.java:84)

      at org.apache.hadoop.hive.ql.security.authorization.plugin.HiveAuthorizerImpl.checkPrivileges(HiveAuthorizerImpl.java:86)

      at org.apache.hadoop.hive.ql.Driver.doAuthorizationV2(Driver.java:1361)

      at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:1125)

      at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:718)

      at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1912)

      at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1859)

      at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1854)

      at org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)

      at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:204)

      at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:269)

      at org.apache.hive.service.cli.operation.Operation.run(Operation.java:268)

      at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:576)

      at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:561)

      at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:315)

      at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:568)

      at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1557)

      at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1542)

      at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)

      at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)

      at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)

      at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)

      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

      at java.lang.Thread.run(Thread.java:748)

      Caused by: java.lang.reflect.UndeclaredThrowableException

      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1700)

      at org.apache.hadoop.hive.common.FileUtils.checkFileAccessWithImpersonation(FileUtils.java:411)

      at org.apache.hadoop.hive.common.FileUtils.isActionPermittedForFileHierarchy(FileUtils.java:471)

      at org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLAuthorizationUtils.getPrivilegesFromFS(SQLAuthorizationUtils.java:457)

      at org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLAuthorizationUtils.addPrivilegesFromFS(SQLAuthorizationUtils.java:444)

      at org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLAuthorizationUtils.getPrivilegesFromFS(SQLAuthorizationUtils.java:417)

      ... 26 more

      Caused by: java.lang.reflect.InvocationTargetException

      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

      at java.lang.reflect.Method.invoke(Method.java:498)

      at org.apache.hadoop.hive.shims.Hadoop23Shims.checkFileAccess(Hadoop23Shims.java:950)

      at org.apache.hadoop.hive.common.FileUtils$3.run(FileUtils.java:415)

      at java.security.AccessController.doPrivileged(Native Method)

      at javax.security.auth.Subject.doAs(Subject.java:422)

      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)

      ... 31 more

      Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: sam.an is not allowed to impersonate hive

      at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1499)

      at org.apache.hadoop.ipc.Client.call(Client.java:1445)

      at org.apache.hadoop.ipc.Client.call(Client.java:1355)

      at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:228)

      at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)

      at com.sun.proxy.$Proxy31.checkAccess(Unknown Source)

      at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.checkAccess(ClientNamenodeProtocolTranslatorPB.java:1714)

      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

      at java.lang.reflect.Method.invoke(Method.java:498)

      at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422)

      at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165)

      at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157)

      at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)

      at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359)

      at com.sun.proxy.$Proxy32.checkAccess(Unknown Source)

      at org.apache.hadoop.hdfs.DFSClient.checkAccess(DFSClient.java:2835)

      at org.apache.hadoop.hdfs.DistributedFileSystem$64.doCall(DistributedFileSystem.java:2794)

      at org.apache.hadoop.hdfs.DistributedFileSystem$64.doCall(DistributedFileSystem.java:2791)

      at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)

      at org.apache.hadoop.hdfs.DistributedFileSystem.access(DistributedFileSystem.java:2804)

      ... 40 more

       

      I am not sure this is a genuine bug or it's a my hadoop set up incorrectly. Logging this ticket to track it to make sure. 

        Attachments

          Activity

            People

            • Assignee:
              samuelan Sam An
              Reporter:
              samuelan Sam An
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: