Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-22860

Spark workers log ssl passwords passed to the executors

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.1.1
    • 3.0.0
    • Spark Core
    • None

    Description

      The workers log the spark.ssl.keyStorePassword and spark.ssl.trustStorePassword passed by cli to the executor processes. The ExecutorRunner should escape passwords to not appear in the worker's log files in INFO level. In this example, you can see my 'SuperSecretPassword' in a worker log:

      17/12/08 08:04:12 INFO ExecutorRunner: Launch command: "/global/myapp/oem/jdk/bin/java" "-cp" "/global/myapp/application/myapp_software/thing_loader_lib/core-repository-model-zzz-1.2.3-SNAPSHOT.jar
      [...]
      :/global/myapp/application/spark-2.1.1-bin-hadoop2.7/jars/*" "-Xmx16384M" "-Dspark.authenticate.enableSaslEncryption=true" "-Dspark.ssl.keyStorePassword=SuperSecretPassword" "-Dspark.ssl.keyStore=/global/myapp/application/config/ssl/keystore.jks" "-Dspark.ssl.trustStore=/global/myapp/application/config/ssl/truststore.jks" "-Dspark.ssl.enabled=true" "-Dspark.driver.port=39927" "-Dspark.ssl.protocol=TLS" "-Dspark.ssl.trustStorePassword=SuperSecretPassword" "-Dspark.authenticate=true" "-Dmyapp_IMPORT_DATE=2017-10-30" "-Dmyapp.config.directory=/global/myapp/application/config" "-Dsolr.httpclient.builder.factory=com.company.myapp.loader.auth.LoaderConfigSparkSolrBasicAuthConfigurer" "-Djavax.net.ssl.trustStore=/global/myapp/application/config/ssl/truststore.jks" "-XX:+UseG1GC" "-XX:+UseStringDeduplication" "-Dthings.loader.export.zzz_files=false" "-Dlog4j.configuration=file:/global/myapp/application/config/spark-executor-log4j.properties" "-XX:+HeapDumpOnOutOfMemoryError" "-XX:+UseStringDeduplication" "org.apache.spark.executor.CoarseGrainedExecutorBackend" "--driver-url" "spark://CoarseGrainedScheduler@192.168.0.1:39927" "--executor-id" "2" "--hostname" "192.168.0.1" "--cores" "4" "--app-id" "app-20171208080412-0000" "--worker-url" "spark://Worker@192.168.0.1:59530"
      

      Attachments

        Issue Links

          Activity

            People

              kabhwan Jungtaek Lim
              QAFelixK Felix K.
              Votes:
              2 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: