Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-20970

Deprecate TaskMetrics._updatedBlockStatuses

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Incomplete
    • 2.2.0
    • None
    • Spark Core

    Description

      TaskMetrics._updatedBlockStatuses isn't used anywhere internally by spark. It could be used by users though since its exposed by SparkListenerTaskEnd. We made it configurable to turn off the tracking of it since it uses a lot of memory in https://issues.apache.org/jira/browse/SPARK-20923. That config is still true for backwards compatibility. We should turn that to false in next release and deprecate that api altogether.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              tgraves Thomas Graves
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: