Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-5758 Load tests for SyntheticSources in Python
  3. BEAM-6335

GroupByKey uses data insertion pipeline in streaming tests

Details

    • Sub-task
    • Status: Resolved
    • P2
    • Resolution: Fixed
    • None
    • Not applicable
    • testing
    • None

    Description

      Uses prepared Java Data Insertion Pipeline to update GroupByKey in Python to load test streaming.

      This task contains following steps:

      1. Create GroupByKey streaming test that accepts bytes
      2. To stop test after arrived messages it is required to add a matcher. The matcher should work on number of messages because in case of load testing it would be difficult to compare big load of bytes (also casting bytes to string to compare load would be difficult).
      3. All data is generated by SyntheticDataPublisher.java, which is sending bytes produced by synthetic source to PubSub. PubSub is used as a streaming source for the Python test.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              kasiak Kasia Kucharczyk
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 50m
                  50m