Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersConvert to IssueMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete CommentsDelete
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Minor
    • Resolution: Resolved
    • 3.1.0
    • None
    • ML
    • None

    Description

      performace test in https://issues.apache.org/jira/browse/SPARK-31783,
      Huber loss seems start to diverge since 70 iters.

       

       for (size <- Seq(1, 4, 16, 64); iter <- Seq(10, 50, 100)) {
          Thread.sleep(10000)
          val hlir = new LinearRegression().setLoss("huber").setSolver("l-bfgs").setMaxIter(iter).setTol(0)
          val start = System.currentTimeMillis
          val model = hlir.setBlockSize(size).fit(df)
          val end = System.currentTimeMillis
          println((model.uid, size, iter, end - start, model.summary.objectiveHistory.last, model.summary.totalIterations, model.coefficients.toString.take(100)))
      }

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned Assign to me
            podongfeng Ruifeng Zheng
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment