Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-16470

ml.regression.LinearRegression training data do not check whether the result actually reach convergence

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Trivial
    • Resolution: Fixed
    • 2.0.0
    • 2.0.0
    • ML, Optimizer
    • None

    Description

      In `ml.regression.LinearRegression`, it use breeze `LBFGS` and `OWLQN` optimizer to do data training, but do not check whether breeze's optimizer returned result actually reached convergence.

      In fact, the `LBFGS` and `OWLQN` optimizer in breeze finish iteration may result in the following situations:

      1) reach max iteration number
      2) function reach value convergence
      3) objective function stop improving
      4) gradient reach convergence
      5) search failed(due to some internal numerical error)

      only (2) & (4) situations above can be regarded as convergence result.
      in other situation, we should at least print some warning to remind spark user.

      Attachments

        Activity

          People

            weichenxu123 Weichen Xu
            weichenxu123 Weichen Xu
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - 24h
                24h
                Remaining:
                Remaining Estimate - 24h
                24h
                Logged:
                Time Spent - Not Specified
                Not Specified