Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-7111

apache_beam.examples.wordcount_it_test.WordCountIT: failure in streaming mode (postcommit)

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.13.0
    • Component/s: sdk-py-core
    • Labels:
      None

      Description

      $ ./scripts/run_integration_test.sh --runner TestDirectRunner --streaming true --test_opts --tests=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
      /usr/local/google/home/ehudm/virtualenvs/beamenv/local/lib/python2.7/site-packages/setuptools/dist.py:475: UserWarning: Normalizing '2.13.0.dev' to '2.13.0.dev0'
      normalized_version,
      warning: no files found matching 'README.md'
      warning: no files found matching 'NOTICE'
      warning: no files found matching 'LICENSE'
      warning: cmd: standard file not found: should have one of README, README.rst, README.txt, README.md

      >>> RUNNING integration tests with pipeline options: --runner=TestDirectRunner --project=apache-beam-testing --staging_location=gs://temp-storage-for-end-to-end-tests/staging-it --temp_location=gs://temp-storage-for-end-to-end-tests/temp-it --output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output --sdk_location=dist/apache-beam-2.13.0.dev0.tar.gz --requirements_file=postcommit_requirements.txt --num_workers=1 --sleep_secs=20 --streaming --kms_key_name=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test --dataflow_kms_key=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test
      >>> test options: --tests=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it
      /usr/local/google/home/ehudm/virtualenvs/beamenv/local/lib/python2.7/site-packages/setuptools/dist.py:475: UserWarning: Normalizing '2.13.0.dev' to '2.13.0.dev0'
      normalized_version,
      running nosetests
      running egg_info
      writing requirements to apache_beam.egg-info/requires.txt
      writing apache_beam.egg-info/PKG-INFO
      writing top-level names to apache_beam.egg-info/top_level.txt
      writing dependency_links to apache_beam.egg-info/dependency_links.txt
      writing entry points to apache_beam.egg-info/entry_points.txt
      reading manifest file 'apache_beam.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      warning: no files found matching 'README.md'
      warning: no files found matching 'NOTICE'
      warning: no files found matching 'LICENSE'
      writing manifest file 'apache_beam.egg-info/SOURCES.txt'
      test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... FAIL

      ======================================================================
      FAIL: test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT)
      ----------------------------------------------------------------------
      Traceback (most recent call last):
      File "/usr/local/google/home/ehudm/src/beam/sdks/python/apache_beam/examples/wordcount_it_test.py", line 47, in test_wordcount_it
      self._run_wordcount_it(wordcount.run)
      File "/usr/local/google/home/ehudm/src/beam/sdks/python/apache_beam/examples/wordcount_it_test.py", line 75, in _run_wordcount_it
      run_wordcount(test_pipeline.get_full_options_as_args(**extra_opts))
      File "/usr/local/google/home/ehudm/src/beam/sdks/python/apache_beam/examples/wordcount.py", line 114, in run
      result = p.run()
      File "/usr/local/google/home/ehudm/src/beam/sdks/python/apache_beam/pipeline.py", line 406, in run
      self._options).run(False)
      File "/usr/local/google/home/ehudm/src/beam/sdks/python/apache_beam/pipeline.py", line 419, in run
      return self.runner.run_pipeline(self, self._options)
      File "/usr/local/google/home/ehudm/src/beam/sdks/python/apache_beam/runners/direct/test_direct_runner.py", line 51, in run_pipeline
      hc_assert_that(self.result, pickler.loads(on_success_matcher))
      AssertionError:
      Expected: (Test pipeline expected terminated in state: DONE and Expected checksum is 33535a832b7db6d78389759577d4ff495980b9c0)
      but: Test pipeline expected terminated in state: DONE Test pipeline job terminated in state: RUNNING

      -------------------- >> begin captured logging << --------------------
      root: INFO: Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt
      root: INFO: Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
      root: DEBUG: Connecting using Google Application Default Credentials.
      oauth2client.transport: INFO: Attempting refresh to obtain initial access_token
      oauth2client.client: INFO: Refreshing access_token
      root: DEBUG: Connecting using Google Application Default Credentials.
      oauth2client.transport: INFO: Attempting refresh to obtain initial access_token
      oauth2client.client: INFO: Refreshing access_token
      apache_beam.io.filesystem: DEBUG: translate_pattern: 'gs://dataflow-samples/shakespeare/kinglear.txt' -> 'gs\\:\\/\\/dataflow\\-samples\\/shakespeare\\/kinglear
      .txt'
      root: INFO: Running pipeline with DirectRunner.
      --------------------- >> end captured logging << ---------------------

      ----------------------------------------------------------------------
      XML: /usr/local/google/home/ehudm/src/beam/sdks/python/nosetests.xml
      ----------------------------------------------------------------------
      Ran 1 test in 1.069s

      FAILED (failures=1)

        Attachments

          Activity

            People

            • Assignee:
              pabloem Pablo Estrada
              Reporter:
              udim Udi Meiri
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: