Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-27777

Eliminate uncessary sliding job in AreaUnderCurve

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 3.0.0
    • 3.0.0
    • ML
    • None

    Description

      Current impl of {AreaUnderCurve} use {SlidingRDD} to perform sliding, in which a prepending job is needed to compute the head items on each partition.

      However, this job can be eliminated in computation of AUC by collecting local areas and head/last at once.

      Attachments

        Issue Links

          Activity

            People

              podongfeng Ruifeng Zheng
              podongfeng Ruifeng Zheng
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: