Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
3.3.0
-
None
-
None
-
CentOS Linux release 7.7.1908 (Core)
Description
I have extended Hadoop (YARN, HDFS, ...) web UIs with basic authentication by implementing a custom authentication handler class and configuring it via `hadoop.http.authentication.type` in core-site.xml.
Everything was working fine with Hadoop 2.x and 3.2.2. However, when I tried switching to 3.3.0 standby Namenode failed during bootstrap when trying to get fsimage:
2021-02-25 21:18:52,356 ERROR [main] org.apache.hadoop.hdfs.server.namenode.NameNode: Failed to start namenode. java.io.IOException: java.lang.RuntimeException: org.apache.hadoop.hdfs.server.common.HttpGetFailedException: Image transfer servlet at http://XXXXX/imagetransfer?ge timage=1&txid=0&storageInfo=-65:271209174:1614287921618:CID-f21dbb8a-8660-4ef6-8045-f80daf067c38&bootstrapstandby=true failed with status code 401 Response message: Authentication required at org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby.run(BootstrapStandby.java:549) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1728) at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1821) Caused by: java.lang.RuntimeException: org.apache.hadoop.hdfs.server.common.HttpGetFailedException: Image transfer servlet at http://XXXXX/imagetransfer?getimage=1&t xid=0&storageInfo=-65:271209174:1614287921618:CID-f21dbb8a-8660-4ef6-8045-f80daf067c38&bootstrapstandby=true failed with status code 401 Response message: Authentication required at org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby$1.run(BootstrapStandby.java:127) at org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby$1.run(BootstrapStandby.java:121) at org.apache.hadoop.security.SecurityUtil.doAsLoginUserOrFatal(SecurityUtil.java:485) at org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby.run(BootstrapStandby.java:121) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90) at org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby.run(BootstrapStandby.java:544) ... 2 more Caused by: org.apache.hadoop.hdfs.server.common.HttpGetFailedException: Image transfer servlet at http://XXXXXX/imagetransfer?getimage=1&txid=0&storageInfo=-65:271209 174:1614287921618:CID-f21dbb8a-8660-4ef6-8045-f80daf067c38&bootstrapstandby=true failed with status code 401 Response message: Authentication required at org.apache.hadoop.hdfs.server.common.Util.doGetUrl(Util.java:168) at org.apache.hadoop.hdfs.server.namenode.TransferFsImage.doGetUrl(TransferFsImage.java:441) at org.apache.hadoop.hdfs.server.namenode.TransferFsImage.getFileClient(TransferFsImage.java:436) at org.apache.hadoop.hdfs.server.namenode.TransferFsImage.downloadImageToStorage(TransferFsImage.java:123) at org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby.downloadImage(BootstrapStandby.java:357) at org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby.doRun(BootstrapStandby.java:239) at org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby.access$000(BootstrapStandby.java:82) at org.apache.hadoop.hdfs.server.namenode.ha.BootstrapStandby$1.run(BootstrapStandby.java:125) ... 8 more
It seems that in 3.3.0 my custom handler was being invoked for `/imagetransfer` endpoint, from active NN:
2021-02-25 21:18:52,328 DEBUG [qtp2033968586-22] org.apache.hadoop.security.authentication.server.AuthenticationFilter: Request [http://XXXXX/imagetransfer?getimage=1&txid=0&storageInfo=-65:271209174:1614287921618:CID-f21dbb8a-8660-4ef6-8045-f80daf067c38&bootstrapstandby=true] triggering authentication. handler: class hadoop_auth_handler.BasicAuthenticationHandler
which was not the case for 3.2.2 and earlier.
Attachments
Issue Links
- is caused by
-
HADOOP-16314 Make sure all end point URL is covered by the same AuthenticationFilter
- Resolved