Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-5758

Load tests for SyntheticSources in Python

    XMLWordPrintableJSON

    Details

    • Type: Test
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: testing
    • Labels:
      None

      Description

      For purpose of load testing the SyntheticSources there should be created tests with metrics which will be sent to BigQuery.

        Attachments

          Issue Links

          1.
          Create ParDo Load Test Sub-task Resolved Kasia Kucharczyk  
          2.
          Create Combine Load Test Sub-task Resolved Kasia Kucharczyk  
          3.
          Create CoGroupByKey Load Test Sub-task Resolved Kasia Kucharczyk  
          4.
          Create GroupByKey Load Test Sub-task Resolved Kasia Kucharczyk  
          5.
          Create SideInput Load test Sub-task Resolved Kasia Kucharczyk

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 1h 40m
          6.
          Publish metrics from load tests to BigQuery database Sub-task Resolved Kasia Kucharczyk

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 2h 50m
          7.
          Create Smoke and GBK Python Load Test Jenkins Job Sub-task Resolved Kasia Kucharczyk

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 53h 10m
          8.
          GroupByKey uses data insertion pipeline in streaming tests Sub-task In Progress Kasia Kucharczyk

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 50m
          9.
          Remove runninig job twice Sub-task Closed Kasia Kucharczyk

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 50m
          10.
          Create ParDo Python Load Test Jenkins Job Sub-task Resolved Kamil Wasilewski

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 8h 10m
          11.
          Create CoGBK Python Load Test Job Jenkins Sub-task Resolved Kamil Wasilewski

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 1h 10m
          12.
          Create Combine Python Load Test Jenkins Job Sub-task Resolved Kamil Wasilewski

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 7h 20m
          13.
          Create SideInput Python Load Test Jenkins Job Sub-task Open Kasia Kucharczyk  
          14.
          Implement missing pipeline parameters in ParDo Load Test Sub-task Resolved Kamil Wasilewski

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 4h 10m
          15.
          Create ParDo Python Load Test Jenkins Job [Flink] Sub-task Resolved Kamil Wasilewski

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 4h 40m
          16.
          Create CoGBK Python Load Test Jenkins Job [Flink] Sub-task Resolved Kamil Wasilewski

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 0.5h
          17.
          Create Combine Python Load Test Jenkins Job [Flink] Sub-task Resolved Kamil Wasilewski

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 3h 40m
          18.
          Create SideInput Python Load Test Jenkins Job [Flink] Sub-task Open Kamil Wasilewski  
          19.
          Add the rest of GBK tests in Jenkins job [Flink] Sub-task Resolved Kasia Kucharczyk

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 10h 40m
          20.
          Simpler Flink cluster set up in load tests Sub-task Resolved Kamil Wasilewski

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 15.5h
          21.
          Do not publish any metrics in Load tests on Flink Sub-task Resolved Kamil Wasilewski

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 0.5h
          22.
          Upgrade supported Flink version to 1.9 Sub-task Resolved Kamil Wasilewski

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 5h 40m
          23.
          Turn on metrics publishing in load tests on Flink Sub-task Resolved Kamil Wasilewski

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 2.5h
          24.
          Migrate Python version to 3.7 Sub-task Open Kamil Wasilewski

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 13h 10m
          25.
          Add stateful processing to ParDo load test Sub-task Open Maximilian Michels

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 1h 40m
          26.
          Bring back the names of the runtime metrics to "runtime" Sub-task Open Unassigned  
          27.
          Create a common place for Load Tests configuration Sub-task Open Pawel Pasterz

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 5h 40m
          28.
          Stop using nose in load tests Sub-task Open Kamil Wasilewski

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 10.5h
          29.
          Include Dataflow Job ID in console output Sub-task Triage Needed Unassigned  

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                kasiak Kasia Kucharczyk
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 157h 50m
                  157h 50m