Details
-
Bug
-
Status: Resolved
-
Minor
-
Resolution: Not A Problem
-
2.4.3
-
None
-
None
Description
Currently the docker images generated by the distro uses openjdk8 based on alpine.
This means that the version shipped of libssl is 1.1.1b-r1:
sh-4.4# apk list | grep ssl libssl1.1-1.1.1b-r1 x86_64 {openssl} (OpenSSL) [installed]
The hadoop distro ships wildfly-openssl-1.0.4.Final.jar, which is affected by https://issues.jboss.org/browse/JBEAP-16425.
This results on error running on the executor:
2019-07-04 22:32:40,339 INFO openssl.SSL: WFOPENSSL0002 OpenSSL Version OpenSSL 1.1.1b 26 Feb 2019 2019-07-04 22:32:40,363 WARN streaming.FileStreamSink: Error while looking for metadata directory. Exception in thread "main" java.lang.NullPointerException at org.wildfly.openssl.CipherSuiteConverter.toJava(CipherSuiteConverter.java:284)
In my tests creating a Docker image with an updated version of wildfly, solves the issue: 1.0.7.Final
Not sure if this is an Spark problem, if so, where would be the right place to solve it.
It seems they may take care of it in Hadoop directly, but tickets are open.
Attachments
Issue Links
- is related to
-
HADOOP-16410 Hadoop 3.2 azure jars incompatible with alpine 3.9
- Resolved