Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-17704

ChiSqSelector performance improvement.

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 2.1.0
    • ML, MLlib
    • None

    Description

      Several performance improvement for ChiSqSelector:
      1, Keep selectedFeatures ordered ascendent.
      ChiSqSelectorModel.transform need selectedFeatures ordered to make prediction. We should sort it when training model rather than making prediction, since users usually train model once and use the model to do prediction multiple times.
      2, When training fpr type ChiSqSelectorModel, it's not necessary to sort the ChiSq test result by statistic.

      Attachments

        Issue Links

          Activity

            People

              yanboliang Yanbo Liang
              yanboliang Yanbo Liang
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: