Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-1981 [Umbrella] Fix all flaky tests
  3. ZEPPELIN-2129

Flaky test - PySparkInterpreterTest fails with TypeError: namedtuple() missing 3 required keyword-only arguments: 'verbose', 'rename', and 'module'

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 0.7.1, 0.8.0
    • None
    • None

    Description

      PysparkInterpreterTest fails with

      https://travis-ci.org/Leemoonsoo/zeppelin/jobs/202563404

      PySparkInterpreterTest.testBasicIntp:111 expected:<SUCCESS> but was:<ERROR>
        PySparkInterpreterTest.testCompletion:120 null
        PySparkInterpreterMatplotlibTest.dependenciesAreInstalled:144 [%text Traceback (most recent call last):
        File "/tmp/zeppelin_pyspark-218141602126326064.py", line 22, in <module>
          from pyspark.conf import SparkConf
        File "<frozen importlib._bootstrap>", line 961, in _find_and_load
        File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
        File "<frozen importlib._bootstrap>", line 646, in _load_unlocked
        File "<frozen importlib._bootstrap>", line 616, in _load_backward_compatible
        File "/home/travis/build/Leemoonsoo/zeppelin/interpreter/spark/pyspark/pyspark.zip/pyspark/__init__.py", line 44, in <module>
        File "<frozen importlib._bootstrap>", line 961, in _find_and_load
        File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
        File "<frozen importlib._bootstrap>", line 646, in _load_unlocked
        File "<frozen importlib._bootstrap>", line 616, in _load_backward_compatible
        File "/home/travis/build/Leemoonsoo/zeppelin/interpreter/spark/pyspark/pyspark.zip/pyspark/context.py", line 37, in <module>
        File "<frozen importlib._bootstrap>", line 961, in _find_and_load
        File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
        File "<frozen importlib._bootstrap>", line 646, in _load_unlocked
        File "<frozen importlib._bootstrap>", line 616, in _load_backward_compatible
        File "/home/travis/build/Leemoonsoo/zeppelin/interpreter/spark/pyspark/pyspark.zip/pyspark/rdd.py", line 47, in <module>
        File "<frozen importlib._bootstrap>", line 961, in _find_and_load
        File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
        File "<frozen importlib._bootstrap>", line 646, in _load_unlocked
        File "<frozen importlib._bootstrap>", line 616, in _load_backward_compatible
        File "/home/travis/build/Leemoonsoo/zeppelin/interpreter/spark/pyspark/pyspark.zip/pyspark/statcounter.py", line 24, in <module>
        File "/home/travis/miniconda/lib/python3.6/site-packages/numpy/__init__.py", line 146, in <module>
          from . import add_newdocs
        File "/home/travis/miniconda/lib/python3.6/site-packages/numpy/add_newdocs.py", line 13, in <module>
          from numpy.lib import add_newdoc
        File "/home/travis/miniconda/lib/python3.6/site-packages/numpy/lib/__init__.py", line 8, in <module>
          from .type_check import *
        File "/home/travis/miniconda/lib/python3.6/site-packages/numpy/lib/type_check.py", line 11, in <module>
          import numpy.core.numeric as _nx
        File "/home/travis/miniconda/lib/python3.6/site-packages/numpy/core/__init__.py", line 72, in <module>
          from numpy.testing.nosetester import _numpy_tester
        File "/home/travis/miniconda/lib/python3.6/site-packages/numpy/testing/__init__.py", line 10, in <module>
          from unittest import TestCase
        File "/home/travis/miniconda/lib/python3.6/unittest/__init__.py", line 58, in <module>
          from .result import TestResult
        File "/home/travis/miniconda/lib/python3.6/unittest/result.py", line 7, in <module>
          from . import util
        File "/home/travis/miniconda/lib/python3.6/unittest/util.py", line 119, in <module>
          _Mismatch = namedtuple('Mismatch', 'actual expected value')
        File "/home/travis/build/Leemoonsoo/zeppelin/interpreter/spark/pyspark/pyspark.zip/pyspark/serializers.py", line 381, in namedtuple
      TypeError: namedtuple() missing 3 required keyword-only arguments: 'verbose', 'rename', and 'module'
      , %text Traceback (most recent call last):
      

      Which is because of pyspark does not work with python 3.6
      https://issues.apache.org/jira/browse/SPARK-19019

      Attachments

        Issue Links

          Activity

            People

              moon Lee Moon Soo
              moon Lee Moon Soo
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: