Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-5996

Nexmark postCommits are failing for Dataflow

Details

    • Improvement
    • Status: Resolved
    • P2
    • Resolution: Fixed
    • None
    • 2.9.0
    • None

    Description

      Here is the gradle build scan: https://scans.gradle.com/s/co2uh5xame2pc

      apilloud I took the liberty to assign it to you as you did the nexmark to dataflow integration. Feel free to re-assign if needed.

       

      java.io.IOException: com.google.api.client.googleapis.json.GoogleJsonResponseException: 410 Gone
      {
        "code" : 429,
        "errors" : [ {
          "domain" : "usageLimits",
          "message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-WjJ-Xb9pjar2Y6llqnj5Zg.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests.",
          "reason" : "rateLimitExceeded"
        } ],
        "message" : "The total number of changes to the object temp-storage-for-perf-tests/nexmark/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.9.0-SNAPSHOT-WjJ-Xb9pjar2Y6llqnj5Zg.jar exceeds the rate limit. Please reduce the rate of create, update, and delete requests."
      }
      at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:432)
      at com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:287)
      at org.apache.beam.runners.dataflow.util.PackageUtil.$closeResource(PackageUtil.java:260)
      at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackage(PackageUtil.java:260)
      at org.apache.beam.runners.dataflow.util.PackageUtil.tryStagePackageWithRetry(PackageUtil.java:203)
      at org.apache.beam.runners.dataflow.util.PackageUtil.stagePackageSynchronously(PackageUtil.java:187)
      at org.apache.beam.runners.dataflow.util.PackageUtil.lambda$stagePackage$1(PackageUtil.java:171)
      at org.apache.beam.sdk.util.MoreFutures.lambda$supplyAsync$0(MoreFutures.java:
      104)
      at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      at java.lang.Thread.run(Thread.java:748)

      Attachments

        Issue Links

          Activity

            People

              apilloud Andrew Pilloud
              echauchot Etienne Chauchot
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h
                  2h