[SPARK-28159] Make the transform natively in ml framework to avoid extra conversion - ASF JIRA

XML

Word

Printable

JSON

It is a long time since ML was released.

However, there are still many TODOs (like in ChiSqSelector.scala

{// TODO: Make the transformer natively in ml framework to avoid extra conversion.}

) on making transform natively in ml framework.

I try to make ml algs no longer need to convert ml-vector to mllib-vector in transforms.

Including: LDA/ChiSqSelector/ElementwiseProduct/HashingTF/IDF/Normalizer/PCA/StandardScaler.

is duplicated by

SPARK-22531 Migrate the implementation of IDF from MLLib to ML

SPARK-24283 Make standard scaler work without legacy MLlib

links to

GitHub Pull Request #24963