Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-9904

Allow users to control MaxDirectMemorySize

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Duplicate
    • 1.4.2, 1.5.1, 1.7.2, 1.8.0, 1.9.0
    • None
    • Deployment / Scripts
    • Patch

    Description

      For people who use docker image and run flink in pods, currently, there is no way to update 

      MaxDirectMemorySize

      (Well one can create a custom version of taskmanager.sh)

       

      As a result, it starts with a value of 8388607T . If the param taskmanager.memory.preallocate is set to false (default) the clean up will only occur when the MaxDirectMemorySize limit is hit and a gc full cycle kicks in. However with pods especially in kuberenete they will get killed because pods do not run at such a high value. (In our case we run 8GB per pod)

       

      The fix would be to allow it be configurable via flink-conf. We can still have a default of 8388607T to avoid a breaking change. 

       

      Attachments

        Issue Links

          Activity

            People

              tianchen92 Ji Liu
              hroongta Himanshu Roongta
              Votes:
              2 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1.5h
                  1.5h