Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
1.0.0, 2.4.0
-
None
-
None
Description
Hiveserver2 needs permission 733 or above on scratch directory to start successfully.
HS2 does not take into consideration the permission given to scratch dir via Ranger, it expects the permissions at HDFS level.
Even if we give full access to 'hive' user from Ranger , the start of HS2 fails, it expects to have the permission from HDFS (#hdfs dfs -chmod 755 /tmp/hive)
>> SessionState.java
private Path createRootHDFSDir(HiveConf conf) throws IOException { Path rootHDFSDirPath = new Path(HiveConf.getVar(conf, HiveConf.ConfVars.SCRATCHDIR)); FsPermission writableHDFSDirPermission = new FsPermission((short)00733); FileSystem fs = rootHDFSDirPath.getFileSystem(conf); if (!fs.exists(rootHDFSDirPath)) { Utilities.createDirsWithPermission(conf, rootHDFSDirPath, writableHDFSDirPermission, true); } FsPermission currentHDFSDirPermission = fs.getFileStatus(rootHDFSDirPath).getPermission(); if (rootHDFSDirPath != null && rootHDFSDirPath.toUri() != null) { String schema = rootHDFSDirPath.toUri().getScheme(); LOG.debug( "HDFS root scratch dir: " + rootHDFSDirPath + " with schema " + schema + ", permission: " + currentHDFSDirPermission); } else { LOG.debug( "HDFS root scratch dir: " + rootHDFSDirPath + ", permission: " + currentHDFSDirPermission); } // If the root HDFS scratch dir already exists, make sure it is writeable. if (!((currentHDFSDirPermission.toShort() & writableHDFSDirPermission .toShort()) == writableHDFSDirPermission.toShort())) { throw new RuntimeException("The root scratch dir: " + rootHDFSDirPath + " on HDFS should be writable. Current permissions are: " + currentHDFSDirPermission); }
>> Error message :
2017-08-23 09:56:13,965 WARN [main]: server.HiveServer2 (HiveServer2.java:startHiveServer2(508)) - Error starting HiveServer2 on attempt 1, will retry in 60 seconds java.lang.RuntimeException: Error applying authorization policy on hive configuration: java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable. Current permissions are: rwxr-x--- at org.apache.hive.service.cli.CLIService.init(CLIService.java:117) at org.apache.hive.service.CompositeService.init(CompositeService.java:59) at org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:122) at org.apache.hive.service.server.HiveServer2.startHiveServer2(HiveServer2.java:474) at org.apache.hive.service.server.HiveServer2.access$700(HiveServer2.java:87) at org.apache.hive.service.server.HiveServer2$StartOptionExecutor.execute(HiveServer2.java:720) at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:593) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:233) at org.apache.hadoop.util.RunJar.main(RunJar.java:148) Caused by: java.lang.RuntimeException: java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable. Current permissions are: rwxr-x--- at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:547) at org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:130) at org.apache.hive.service.cli.CLIService.init(CLIService.java:115) ... 12 more Caused by: java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable. Current permissions are: rwxr-x--- at org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:648) at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:580) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:533) ... 14 more