Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-9049

MemoryMonitor thrashing detection is too aggressive for batch workers

Details

    • Improvement
    • Status: Open
    • P3
    • Resolution: Unresolved
    • None
    • None
    • runner-dataflow
    • None

    Description

      In the streaming dataflow worker we've implemented push-back so that we will reduce parallelism when there is memory pressure. Since we cannot do this on the batch worker, it doesn't make sense for our thrashing detection to be so aggressive. We should increase the thresholds used for thrashing detection when running on Batch workloads.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              bhulette Brian Hulette
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 50m
                  1h 50m