Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-10852

Combine Python streaming load test is too slow on Flink

Details

    • Bug
    • Status: Open
    • P3
    • Resolution: Unresolved
    • None
    • None
    • None

    Description

      One of the Combine load test cases, which involves a global combiner and data stream of 200M elements, takes too long on Flink. Flink is able to process only a half of that data stream within 1 hour, which is too long for a Jenkins job.

      Job's definition: https://github.com/apache/beam/blob/master/.test-infra/jenkins/job_LoadTests_Combine_Flink_Python.groovy#L36

      Test pipeline: https://github.com/apache/beam/blob/master/sdks/python/apache_beam/testing/load_tests/combine_test.py

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              kamilwu Kamil Wasilewski
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: