Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-30504

OneVsRest and OneVsRestModel _from_java and _to_java should handle weightCol

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 3.0.0
    • 3.0.0
    • ML, PySpark
    • None

    Description

      Current behaviour

      from pyspark.ml.classification import LogisticRegression, OneVsRest, OneVsRestModel
      from pyspark.ml.linalg import DenseVector
      
      df = spark.createDataFrame([(0, 1, DenseVector([1.0, 0.0])), (0, 1, DenseVector([1.0, 0.0]))], ("label", "w", "features"))
      
      ovr = OneVsRest(classifier=LogisticRegression()).setWeightCol("w")
      ovrm = ovr.fit(df)
      ovr.getWeightCol()
      ## 'w'
      ovrm.getWeightCol()
      ## 'w'
      
      ovr.write().overwrite().save("/tmp/ovr")
      ovr_ = OneVsRest.load("/tmp/ovr")
      ovr_.getWeightCol()
      ## KeyError   
      ## ...
      ## KeyError: Param(parent='OneVsRest_5145d56b6bd1', name='weightCol', doc='weight column name. ...)
      
      ovrm.write().overwrite().save("/tmp/ovrm")
      ovrm_ = OneVsRestModel.load("/tmp/ovrm")
      ovrm_ .getWeightCol()
      ## KeyError   
      ## ...
      ## KeyError: Param(parent='OneVsRestModel_598c6d900fad', name='weightCol', doc='weight column name ...
      

      Expected behaviour:

      OneVsRest and OneVsRestModel loaded from disk should have weightCol.

      Attachments

        Issue Links

          Activity

            People

              zero323 Maciej Szymkiewicz
              zero323 Maciej Szymkiewicz
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: