Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-37814 Migrating from log4j 1 to log4j 2
  3. SPARK-37887

PySpark shell sets log level to INFO by default

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.3.0
    • 3.3.0
    • PySpark, Spark Shell
    • None

    Description

      ./bin/pyspark
      
      Python 3.9.5 (default, May 18 2021, 12:31:01)
      [Clang 10.0.0 ] :: Anaconda, Inc. on darwin
      Type "help", "copyright", "credits" or "license" for more information.
      22/01/13 10:28:15 INFO HiveConf: Found configuration file null
      22/01/13 10:28:15 INFO SparkContext: Running Spark version 3.3.0-SNAPSHOT
      ...
      >>> spark.range(10)
      22/01/13 10:31:48 INFO SharedState: Setting hive.metastore.warehouse.dir ('null') to the value of spark.sql.warehouse.dir.
      ...
      

      Attachments

        Issue Links

          Activity

            People

              viirya L. C. Hsieh
              gurwls223 Hyukjin Kwon
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: