Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-42471 Distributed ML <> spark connect
  3. SPARK-43516

Basic estimator / transformer / model / evaluator interfaces and basic transformer / evaluator implementation

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.5.0
    • 3.5.0
    • Connect, ML, PySpark
    • None

    Description

      • Define basic interfaces of Evaluator / Transformer / Model / Evaluator, these interfaces are designed to support both spark connect and legacy spark mode, and are designed to support train / transform / evaluate over either spark dataframe or pandas dataframe
      • Implement a feature transformer MaxAbsScaler, ScalerScaler
      • Implement a regressor evaluator RegressorEvaluator that supports MSE and R2 metric evaluation

      Attachments

        Activity

          People

            weichenxu123 Weichen Xu
            weichenxu123 Weichen Xu
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: