Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-10863

CGroupElasticMemoryController is not work

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • 3.3.1
    • None
    • nodemanager

    Description

      When following the documentation configuring elastic memory resource control, yarn.nodemanager.elastic-memory-control.enabled set true,  yarn.nodemanager.resource.memory.enforced set to false, yarn.nodemanager.pmem-check-enabled set true, and yarn.nodemanager.resource.memory.enabled set true to use cgroup control memory, but elastic memory control is not work.

      I see the code ContainersMonitorImpl.java, in checkLimit function, the skip logic have some problem.  The return condition is strictMemoryEnforcement is true and elasticMemoryEnforcement is false. So, following the document set use elastic memory control, the check logic will continue, when container memory used over limit will killed by checkLimit. 

      if (strictMemoryEnforcement && !elasticMemoryEnforcement) {
        // When cgroup-based strict memory enforcement is used alone without
        // elastic memory control, the oom-kill would take care of it.
        // However, when elastic memory control is also enabled, the oom killer
        // would be disabled at the root yarn container cgroup level (all child
        // cgroups would inherit that setting). Hence, we fall back to the
        // polling-based mechanism.
        return;
      }
      
      

      Attachments

        1. YARN-10863.001-1.patch
          4 kB
          LuoGe
        2. YARN-10863.002.patch
          13 kB
          LuoGe
        3. YARN-10863.004.patch
          14 kB
          LuoGe
        4. YARN-10863.005.patch
          14 kB
          LuoGe
        5. YARN-10863.006.patch
          14 kB
          LuoGe
        6. YARN-10863.007.patch
          14 kB
          LuoGe

        Issue Links

          Activity

            People

              Unassigned Unassigned
              LuoGe LuoGe
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h 10m
                  2h 10m