Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-9150

beam_PostRelease_Python_Candidate fails

Details

    • Bug
    • Status: Open
    • P3
    • Resolution: Unresolved
    • None
    • None
    • build-system
    • None

    Description

      https://builds.apache.org/job/beam_PostRelease_Python_Candidate/171/timestamps/?time=HH:mm:ss&timeZone=GMT-8&appendLog&locale=en_US

      Some interesting lines:

      17:48:36  release/src/main/python-release/run_release_candidate_python_quickstart.sh: line 144: 20935 Killed                  python -m apache_beam.examples.streaming_wordcount --input_topic projects/$PROJECT_ID/topics/$PUBSUB_TOPIC1 --output_topic projects/$PROJECT_ID/topics/$PUBSUB_TOPIC2 --streaming
      
      18:05:25  ############################################################################
      18:05:25  Running HourlyTeamScore example with DirectRunner
      18:05:25  ############################################################################
      18:05:25  /tmp/tmp.gnpMNIFhcL/temp_virtualenv_python2.7/local/lib/python2.7/site-packages/apache_beam/__init__.py:84: UserWarning: You are using Apache Beam with Python 2. New releases of Apache Beam will soon support Python 3 only.
      18:05:25    'You are using Apache Beam with Python 2. '
      18:05:33  INFO:root:Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner.
      18:05:33  INFO:root:Setting socket default timeout to 60 seconds.
      18:05:33  INFO:root:socket default timeout is 60.0 seconds.
      18:05:33  INFO:oauth2client.transport:Attempting refresh to obtain initial access_token
      18:05:34  /tmp/tmp.gnpMNIFhcL/temp_virtualenv_python2.7/local/lib/python2.7/site-packages/apache_beam/io/gcp/bigquery.py:1220: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported
      18:05:34    experiments = p.options.view_as(DebugOptions).experiments or []
      18:05:37  INFO:root:==================== <function annotate_downstream_side_inputs at 0x7f131dc46f50> ====================
      18:05:37  INFO:root:==================== <function fix_side_input_pcoll_coders at 0x7f131dc3e0c8> ====================
      18:05:37  INFO:root:==================== <function lift_combiners at 0x7f131dc3e140> ====================
      18:05:37  INFO:root:==================== <function expand_sdf at 0x7f131dc3e1b8> ====================
      18:05:37  INFO:root:==================== <function expand_gbk at 0x7f131dc3e230> ====================
      18:05:37  INFO:root:==================== <function sink_flattens at 0x7f131dc3e320> ====================
      18:05:37  INFO:root:==================== <function greedily_fuse at 0x7f131dc3e398> ====================
      18:05:37  INFO:root:==================== <function read_to_impulse at 0x7f131dc3e410> ====================
      18:05:37  INFO:root:==================== <function impulse_to_input at 0x7f131dc3e488> ====================
      18:05:37  INFO:root:==================== <function inject_timer_pcollections at 0x7f131dc3e5f0> ====================
      18:05:37  INFO:root:==================== <function sort_stages at 0x7f131dc3e668> ====================
      18:05:37  INFO:root:==================== <function window_pcollection_coders at 0x7f131dc3e6e0> ====================
      18:05:37  INFO:root:Creating state cache with size 100
      18:05:37  INFO:root:Created Worker handler <apache_beam.runners.portability.fn_api_runner.EmbeddedWorkerHandler object at 0x7f12fc89c290> for environment urn: "beam:env:embedded_python:v1"
      18:05:37  
      18:05:37  INFO:root:Running ((ref_AppliedPTransform_ReadInputText/Read/_SDFBoundedSourceWrapper/Impulse_5)+(ReadInputText/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/PairWithRestriction))+((ReadInputText/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/SplitAndSizeRestriction)+(ref_PCollection_PCollection_1_split/Write))
      18:05:38  INFO:root:Running (ref_PCollection_PCollection_1_split/Read)+((((ReadInputText/Read/_SDFBoundedSourceWrapper/ParDo(SDFBoundedSourceDoFn)/Process)+((ref_AppliedPTransform_HourlyTeamScore/ParseGameEventFn_8)+((ref_AppliedPTransform_HourlyTeamScore/FilterStartTime_9)+((ref_AppliedPTransform_HourlyTeamScore/FilterEndTime_10)+((ref_AppliedPTransform_HourlyTeamScore/AddEventTimestamps_11)+((ref_AppliedPTransform_HourlyTeamScore/FixedWindowsTeam_12)+(ref_AppliedPTransform_HourlyTeamScore/ExtractAndSumScore/Map(<lambda at hourly_team_score.py:142>)_14)))))))+(HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/Precombine))+(HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/Group/Write))
      18:05:39  INFO:root:Running ((HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/Group/Read)+(HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/Merge))+((HourlyTeamScore/ExtractAndSumScore/CombinePerKey(sum)/ExtractOutputs)+((ref_AppliedPTransform_TeamScoresDict_22)+((ref_AppliedPTransform_WriteTeamScoreSums/ConvertToRow_24)+((ref_AppliedPTransform_WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/AppendDestination_27)+((ref_AppliedPTransform_WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/AddInsertIds_28)+((ref_AppliedPTransform_WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/AddRandomKeys_30)+((ref_AppliedPTransform_WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/Map(reify_timestamps)_32)+(WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Write))))))))
      18:05:39  INFO:root:Running (WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/GroupByKey/Read)+(((ref_AppliedPTransform_WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/ReshufflePerKey/FlatMap(restore_timestamps)_37)+(ref_AppliedPTransform_WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/CommitInsertIds/RemoveRandomKeys_38))+(ref_AppliedPTransform_WriteTeamScoreSums/WriteToBigQuery/_StreamToBigQuery/StreamInsertRows/ParDo(BigQueryWriteFn)_40))
      18:05:40  bq head -n 500 $DATASET.hourly_team_score_python_$1
      18:05:43  Did not find team scores, retry left: 2
      18:05:58  bq head -n 500 $DATASET.hourly_team_score_python_$1
      18:06:00  Did not find team scores, retry left: 1
      18:06:15  bq head -n 500 $DATASET.hourly_team_score_python_$1
      18:06:17  Did not find team scores, retry left: 0
      18:06:32  FAILED: HourlyTeamScore example failed running on direct-runner.         Did not found scores of team AntiqueBrassPlatypus in beam_postrelease_mobile_gaming.leader_board
      18:06:32  ############################################################################
      18:06:32  Validation FAILED
      18:06:32  ############################################################################
      

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              udim Udi Meiri
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h 10m
                  2h 10m