Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-14283

Reduce number of threads spawned by BigqueryIO StreamingInserts

Details

    • Improvement
    • Status: Triage Needed
    • P2
    • Resolution: Fixed
    • None
    • 2.39.0
    • io-java-gcp
    • None

    Description

      BigqueryIO's BoundedExecutorService is creating many new threads. It can be optimized by acquiring the involved semaphore before task submission instead of after inside the runnable.

      ref:https://github.com/apache/beam/blob/5603bd310a63fa35082e6bc6b06c7fecc90e77d7/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryServicesImpl.java#L1676

      Attachments

        Issue Links

          Activity

            People

              arunpandianp Arun Pandian
              arunpandianp Arun Pandian
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 20m
                  1h 20m