Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-11547

Nexmark ratelimiting mode doesn't work

Details

    • Bug
    • Status: Triage Needed
    • P3
    • Resolution: Unresolved
    • 2.24.0, 2.25.0, 2.26.0
    • None
    • testing-nexmark

    Description

      When using the --isRateLimited option for Nexmark, the system is supposed to hold events back until they are supposed to be sent, in accordance to the event generations rule.

      What happens today is that the events are pushed into the backlog, but are never actually replayed. This leads to only a few events being sent.

      When I tested this with the direct runner and flink I saw that the generator got resumed all the time, and when it gets resumed the backlog is not restored. I hacked together a solution for this locally, but even with the queue collected, most events still lack. From my tests only 1/10 make it through.

      I don't really know the nexmark code well enough to benchmark anymore beyond this.

      Attachments

        Activity

          People

            Unassigned Unassigned
            rHermes Teodor Spæren
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: