org.apache.beam.sdk.io.gcp.datastore.RampupThrottlingFnTest.testRampupThrottler is flaky

Error Message
java.lang.AssertionError: unexpected exception type thrown; expected:<org.apache.beam.sdk.io.gcp.datastore.RampupThrottlingFnTest.RampupDelayException> but was:<org.mockito.exceptions.verification.TooManyActualInvocations>
Stacktrace
java.lang.AssertionError: unexpected exception type thrown; expected:<org.apache.beam.sdk.io.gcp.datastore.RampupThrottlingFnTest.RampupDelayException> but was:<org.mockito.exceptions.verification.TooManyActualInvocations>
	at org.junit.Assert.assertThrows(Assert.java:1020)
	at org.junit.Assert.assertThrows(Assert.java:981)
	at org.apache.beam.sdk.io.gcp.datastore.RampupThrottlingFnTest.testRampupThrottler(RampupThrottlingFnTest.java:83)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
	at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
	at sun.reflect.GeneratedMethodAccessor153.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
	at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:119)
	at sun.reflect.GeneratedMethodAccessor152.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:182)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:164)
	at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:414)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.mockito.exceptions.verification.TooManyActualInvocations: 
mockCounter.inc(1034L);
Wanted 1 time:
-> at org.apache.beam.sdk.io.gcp.datastore.RampupThrottlingFnTest.lambda$new$0(RampupThrottlingFnTest.java:45)
But was 2 times:
-> at org.apache.beam.sdk.io.gcp.datastore.RampupThrottlingFn.processElement(RampupThrottlingFn.java:119)
-> at org.apache.beam.sdk.io.gcp.datastore.RampupThrottlingFn.processElement(RampupThrottlingFn.java:119)


	at org.apache.beam.sdk.io.gcp.datastore.RampupThrottlingFnTest.lambda$new$0(RampupThrottlingFnTest.java:45)
	at org.apache.beam.sdk.io.gcp.datastore.RampupThrottlingFn.processElement(RampupThrottlingFn.java:120)
	at org.apache.beam.sdk.transforms.DoFnTester.processWindowedElement(DoFnTester.java:216)
	at org.apache.beam.sdk.transforms.DoFnTester.processTimestampedElement(DoFnTester.java:202)
	at org.apache.beam.sdk.transforms.DoFnTester.processElement(DoFnTester.java:195)
	at org.apache.beam.sdk.io.gcp.datastore.RampupThrottlingFnTest.lambda$testRampupThrottler$1(RampupThrottlingFnTest.java:83)
	at org.junit.Assert.assertThrows(Assert.java:1001)
	at org.junit.Assert.assertThrows(Assert.java:981)
	at org.apache.beam.sdk.io.gcp.datastore.RampupThrottlingFnTest.testRampupThrottler(RampupThrottlingFnTest.java:83)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
	at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:119)
Standard Error
Sep 08, 2021 6:35:52 AM org.apache.beam.sdk.transforms.DoFnTester of
WARNING: Your tests use DoFnTester, which may not exercise DoFns correctly. Please use TestPipeline instead.
Sep 08, 2021 6:35:52 AM org.apache.beam.sdk.io.gcp.datastore.RampupThrottlingFn processElement
INFO: Delaying by 771ms to conform to gradual ramp-up.
Sep 08, 2021 6:35:52 AM org.apache.beam.sdk.io.gcp.datastore.RampupThrottlingFn processElement
INFO: Delaying by 1052ms to conform to gradual ramp-up.
Sep 08, 2021 6:35:52 AM org.apache.beam.sdk.io.gcp.datastore.RampupThrottlingFn processElement
INFO: Delaying by 982ms to conform to gradual ramp-up.
Sep 08, 2021 6:35:52 AM org.apache.beam.sdk.io.gcp.datastore.RampupThrottlingFn processElement
INFO: Delaying by 1076ms to conform to gradual ramp-up.
Sep 08, 2021 6:35:52 AM org.apache.beam.sdk.io.gcp.datastore.RampupThrottlingFn processElement
INFO: Delaying by 1034ms to conform to gradual ramp-up.
Sep 08, 2021 6:35:52 AM org.apache.beam.sdk.io.gcp.datastore.RampupThrottlingFn processElement
INFO: Delaying by 1341ms to conform to gradual ramp-up.
Sep 08, 2021 6:35:53 AM org.apache.beam.sdk.io.gcp.datastore.RampupThrottlingFn processElement
INFO: Delaying by 1474ms to conform to gradual ramp-up.
Sep 08, 2021 6:35:53 AM org.apache.beam.sdk.io.gcp.datastore.RampupThrottlingFn processElement
INFO: Delaying by 1096ms to conform to gradual ramp-up.
Sep 08, 2021 6:35:53 AM org.apache.beam.sdk.io.gcp.datastore.RampupThrottlingFn processElement
INFO: Delaying by 1034ms to conform to gradual ramp-up.