Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-26646

Flaky test: pyspark.mllib.tests.test_streaming_algorithms StreamingLogisticRegressionWithSGDTests.test_training_and_prediction

    XMLWordPrintableJSON

Details

    • Test
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.4.7, 3.0.0
    • 2.4.8, 3.0.0
    • MLlib, PySpark
    • None

    Description

      https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101356/console
      https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101358/console
      https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/101254/console
      https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/100941/console
      https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/100327/console

      ======================================================================
      FAIL: test_training_and_prediction (pyspark.mllib.tests.test_streaming_algorithms.StreamingLogisticRegressionWithSGDTests)
      Test that the model improves on toy data with no. of batches
      ----------------------------------------------------------------------
      Traceback (most recent call last):
        File "/home/jenkins/workspace/SparkPullRequestBuilder/python/pyspark/mllib/tests/test_streaming_algorithms.py", line 367, in test_training_and_prediction
          self._eventually(condition, timeout=60.0)
        File "/home/jenkins/workspace/SparkPullRequestBuilder/python/pyspark/mllib/tests/test_streaming_algorithms.py", line 69, in _eventually
          lastValue = condition()
        File "/home/jenkins/workspace/SparkPullRequestBuilder/python/pyspark/mllib/tests/test_streaming_algorithms.py", line 362, in condition
          self.assertGreater(errors[1] - errors[-1], 0.3)
      AssertionError: -0.070000000000000062 not greater than 0.3
      
      ----------------------------------------------------------------------
      Ran 13 tests in 198.327s
      
      FAILED (failures=1, skipped=1)
      
      Had test failures in pyspark.mllib.tests.test_streaming_algorithms with python3.4; see logs.
      

      It apparently became less flaky after increasing the time at SPARK-26275 but looks now it became flacky due to unexpected results.

      Attachments

        Issue Links

          Activity

            People

              viirya L. C. Hsieh
              gurwls223 Hyukjin Kwon
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: