Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-21178

Add support for label specific metrics in MulticlassClassificationEvaluator

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 2.1.1
    • None
    • ML
    • Patch, Important

    Description

      MulticlassClassificationEvaluator is restricted to the global metrics - f1, weightedPrecision, weightedRecall, accuracy

      However, we have a requirement where we would want to optimize the learning on metric for a specific label - for instance, true positive rate (label 'B')

      For example : Take a fraud detection use-case with labels 'good' and 'fraud' being passed to a manual verification team. We want to maximize the true-positive rate of ('fraud') label, so that whenever the model predicts a data point as 'good', it has a strong likelihood of it being 'good', and the manual team can ignore it.
      While it's ok to predict some 'good' data points as 'fraud', as it will be taken care by the manual verification team.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              rawataaryan09 Aman Rawat
              Votes:
              2 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: