Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-26646

Flaky test: pyspark.mllib.tests.test_streaming_algorithms StreamingLogisticRegressionWithSGDTests.test_training_and_prediction

    XMLWordPrintableJSON

    Details

    • Type: Test
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0
    • Fix Version/s: 3.0.0
    • Component/s: MLlib, PySpark
    • Labels:
      None
    • Target Version/s:

      Description

      https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101356/console
      https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101358/console
      https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101254/console
      https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/100941/console
      https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/100327/console

      ======================================================================
      FAIL: test_training_and_prediction (pyspark.mllib.tests.test_streaming_algorithms.StreamingLogisticRegressionWithSGDTests)
      Test that the model improves on toy data with no. of batches
      ----------------------------------------------------------------------
      Traceback (most recent call last):
        File "/home/jenkins/workspace/SparkPullRequestBuilder/python/pyspark/mllib/tests/test_streaming_algorithms.py", line 367, in test_training_and_prediction
          self._eventually(condition, timeout=60.0)
        File "/home/jenkins/workspace/SparkPullRequestBuilder/python/pyspark/mllib/tests/test_streaming_algorithms.py", line 69, in _eventually
          lastValue = condition()
        File "/home/jenkins/workspace/SparkPullRequestBuilder/python/pyspark/mllib/tests/test_streaming_algorithms.py", line 362, in condition
          self.assertGreater(errors[1] - errors[-1], 0.3)
      AssertionError: -0.070000000000000062 not greater than 0.3
      
      ----------------------------------------------------------------------
      Ran 13 tests in 198.327s
      
      FAILED (failures=1, skipped=1)
      
      Had test failures in pyspark.mllib.tests.test_streaming_algorithms with python3.4; see logs.
      

      It apparently became less flaky after increasing the time at SPARK-26275 but looks now it became flacky due to unexpected results.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                viirya L. C. Hsieh
                Reporter:
                hyukjin.kwon Hyukjin Kwon
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: