Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-17715

Log INFO per task launch creates a large driver log

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Trivial
    • Resolution: Fixed
    • None
    • 2.1.0
    • Scheduler, Spark Core
    • None

    Description

      CoarseGrainedSchedulerBackend is logging an INFO every time a task is launched. Previously, only executor-related events were logged as INFO.

      I propose dropping the task logging to a lower level, as task launches can happen orders of magnitude more than executor registration. (I've found the executor logs very useful, so I don't want to block out both via log4j configuration.)

      Attachments

        Activity

          People

            chobrian Brian Cho
            chobrian Brian Cho
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: