Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Duplicate
-
2.7.1
-
None
-
None
-
None
Description
When launching hadoop, the setting of environment variable HADOOP_HEAPMAX results in multiple Xmx parameters being passed to the java application launcher. This is because a default Xmx setting included in the HADOOP_CLIENT_OPTS in etc/hadoop/hadoop-env.sh, line 61:
# The following applies to multiple commands (fs, dfs, fsck, distcp etc) export HADOOP_CLIENT_OPTS="-Xmx512m $HADOOP_CLIENT_OPTS"
is merged into the variable HADOOP_OPTS in bin/hadoop, line 160:
# Always respect HADOOP_OPTS and HADOOP_CLIENT_OPTS HADOOP_OPTS="$HADOOP_OPTS $HADOOP_CLIENT_OPTS"
Orthogonally, the HADOOP_HEAPMAX environment variable triggers the setting of JAVA_HEAP_MAX in hadoop-config.sh, line 192:
# check envvars which might override default args if [ "$HADOOP_HEAPSIZE" != "" ]; then #echo "run with heapsize $HADOOP_HEAPSIZE" JAVA_HEAP_MAX="-Xmx""$HADOOP_HEAPSIZE""m" #echo $JAVA_HEAP_MAX fi
Which is ultimately passed to the java application launcher in bin/hadoop, line 166:
exec "$JAVA" $JAVA_HEAP_MAX $HADOOP_OPTS $CLASS "$@"
The result: two Xmx settings passed to the Java application launcher. Which one is respected depends on the JVM, but for HotSpot the rightmost setting is respected, and so the HADOOP_HEAPMAX environment variable has no effect and the default setting in HADOOP_CLIENT_OPTS wins.
Attachments
Issue Links
- duplicates
-
HADOOP-9902 Shell script rewrite
- Resolved