Details
-
Bug
-
Status: Resolved
-
Critical
-
Resolution: Fixed
-
3.3.0
-
None
-
None
-
None
Description
When starting MiniYARNCluster one could get the following exception:
java.lang.NullPointerException: at org.apache.hadoop.yarn.server.nodemanager.webapp.WebServer.serviceStart(WebServer.java:72) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194) at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:122) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194) at org.apache.hadoop.yarn.server.MiniYARNCluster$NodeManagerWrapper.serviceStart(MiniYARNCluster.java:616) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194) at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:122) at org.apache.hadoop.yarn.server.MiniYARNCluster.serviceStart(MiniYARNCluster.java:327) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194) at org.apache.spark.deploy.yarn.BaseYarnClusterSuite.beforeAll(BaseYarnClusterSuite.scala:96) ...
Looking into the code, this is because we explicitly exclude resource files under hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/resources/TERMINAL, and therefore this code in WebServer fails with NPE:
terminalParams.put("resourceBase", WebServer.class .getClassLoader().getResource("TERMINAL").toExternalForm());
Those who use hadoop-minicluster may not be affected because they'll also need hadoop-yarn-server-nodemanager as an extra dependency, which includes the resource files. On the other hand hadoop-client-minicluster packages both test classes (e.g., MiniYARNCluster) as well as main classes (e.g., ResourceManager and NodeManager) into a single shaded jar. It should include these resource files for testing as well. Otherwise, MiniYARNCluster is unusable.
Attachments
Issue Links
- is broken by
-
YARN-8914 Add xtermjs to YARN UI2
- Resolved
- is duplicated by
-
HADOOP-17324 Don't relocate org.bouncycastle in shaded client jars
- Resolved