Uploaded image for project: 'Mesos'
  1. Mesos
  2. MESOS-7234

NestedMesosContainerizerTest.ROOT_CGROUPS_LaunchNested test is flaky

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Accepted
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • Fedora 24

    Description

      [==========] Running 1 test from 1 test case.
      [----------] Global test environment set-up.
      [----------] 1 test from NestedMesosContainerizerTest
      [ RUN      ] NestedMesosContainerizerTest.ROOT_CGROUPS_LaunchNested
      I0313 09:39:40.803444  1701 containerizer.cpp:221] Using isolation: cgroups/cpu,filesystem/linux,namespaces/pid,network/cni,volume/image
      I0313 09:39:40.811974  1701 linux_launcher.cpp:150] Using /sys/fs/cgroup/freezer as the freezer hierarchy for the Linux launcher
      I0313 09:39:40.812714  1701 provisioner.cpp:249] Using default backend 'overlay'
      I0313 09:39:40.827086  1739 containerizer.cpp:608] Recovering containerizer
      I0313 09:39:40.829702  1737 provisioner.cpp:410] Provisioner recovery complete
      I0313 09:39:40.830343  1744 containerizer.cpp:1001] Starting container 2323db69-fc07-4d2d-bad4-3c7ddce65cd8 for executor 'executor' of framework
      I0313 09:39:40.834182  1738 cpu.cpp:101] Updated 'cpu.shares' to 1024 (cpus 1) for container 2323db69-fc07-4d2d-bad4-3c7ddce65cd8
      I0313 09:39:40.836853  1735 linux_launcher.cpp:429] Launching container 2323db69-fc07-4d2d-bad4-3c7ddce65cd8 and cloning with namespaces CLONE_NEWNS | CLONE_NEWPID
      I0313 09:39:40.861481  1734 containerizer.cpp:1598] Checkpointing container's forked pid 1856 to '/tmp/NestedMesosContainerizerTest_ROOT_CGROUPS_LaunchNested_Sf4Iv8/meta/slaves/frameworks/executors/execut
      or/runs/2323db69-fc07-4d2d-bad4-3c7ddce65cd8/pids/forked.pid'
      I0313 09:39:40.867455  1733 containerizer.cpp:1766] Starting nested container 2323db69-fc07-4d2d-bad4-3c7ddce65cd8.de4bf594-02fe-4ecd-a1b1-6a661b77fe7e
      I0313 09:39:40.870790  1742 linux_launcher.cpp:429] Launching nested container 2323db69-fc07-4d2d-bad4-3c7ddce65cd8.de4bf594-02fe-4ecd-a1b1-6a661b77fe7e and cloning with namespaces CLONE_NEWNS | CLONE_NEW
      PID
      I0313 09:39:45.173310  1737 containerizer.cpp:2483] Container 2323db69-fc07-4d2d-bad4-3c7ddce65cd8.de4bf594-02fe-4ecd-a1b1-6a661b77fe7e has exited
      I0313 09:39:45.173354  1737 containerizer.cpp:2077] Destroying container 2323db69-fc07-4d2d-bad4-3c7ddce65cd8.de4bf594-02fe-4ecd-a1b1-6a661b77fe7e in RUNNING state
      I0313 09:39:45.173630  1731 linux_launcher.cpp:505] Asked to destroy container 2323db69-fc07-4d2d-bad4-3c7ddce65cd8.de4bf594-02fe-4ecd-a1b1-6a661b77fe7e
      I0313 09:39:45.174485  1731 linux_launcher.cpp:548] Using freezer to destroy cgroup mesos_test_03c9934d-4d27-408f-8476-ee1c1983ff1b/2323db69-fc07-4d2d-bad4-3c7ddce65cd8/mesos/de4bf594-02fe-4ecd-a1b1-6a661
      b77fe7e
      I0313 09:39:45.177196  1744 cgroups.cpp:2692] Freezing cgroup /sys/fs/cgroup/freezer/mesos_test_03c9934d-4d27-408f-8476-ee1c1983ff1b/2323db69-fc07-4d2d-bad4-3c7ddce65cd8/mesos/de4bf594-02fe-4ecd-a1b1-6a66
      1b77fe7e
      I0313 09:39:45.179316  1736 cgroups.cpp:1405] Successfully froze cgroup /sys/fs/cgroup/freezer/mesos_test_03c9934d-4d27-408f-8476-ee1c1983ff1b/2323db69-fc07-4d2d-bad4-3c7ddce65cd8/mesos/de4bf594-02fe-4ecd
      -a1b1-6a661b77fe7e after 2.063104ms
      I0313 09:39:45.181565  1733 cgroups.cpp:2710] Thawing cgroup /sys/fs/cgroup/freezer/mesos_test_03c9934d-4d27-408f-8476-ee1c1983ff1b/2323db69-fc07-4d2d-bad4-3c7ddce65cd8/mesos/de4bf594-02fe-4ecd-a1b1-6a661
      b77fe7e
      I0313 09:39:45.183686  1746 cgroups.cpp:1434] Successfully thawed cgroup /sys/fs/cgroup/freezer/mesos_test_03c9934d-4d27-408f-8476-ee1c1983ff1b/2323db69-fc07-4d2d-bad4-3c7ddce65cd8/mesos/de4bf594-02fe-4ec
      d-a1b1-6a661b77fe7e after 2.074112ms
      I0313 09:39:45.187661  1738 containerizer.cpp:2356] Checkpointing termination state to nested container's runtime directory '/tmp/NestedMesosContainerizerTest_ROOT_CGROUPS_LaunchNested_kUGOVz/containers/2
      323db69-fc07-4d2d-bad4-3c7ddce65cd8/containers/de4bf594-02fe-4ecd-a1b1-6a661b77fe7e/termination'
      ../../src/tests/containerizer/nested_mesos_containerizer_tests.cpp:230: Failure
      Expecting WIFEXITED(wait.get()->status()) but  WIFSIGNALED(wait.get()->status()) is true and WTERMSIG(wait.get()->status()) is Killed
      I0313 09:39:45.188985  1737 containerizer.cpp:2077] Destroying container 2323db69-fc07-4d2d-bad4-3c7ddce65cd8 in RUNNING state
      I0313 09:39:45.189237  1734 linux_launcher.cpp:505] Asked to destroy container 2323db69-fc07-4d2d-bad4-3c7ddce65cd8
      I0313 09:39:45.189946  1734 linux_launcher.cpp:548] Using freezer to destroy cgroup mesos_test_03c9934d-4d27-408f-8476-ee1c1983ff1b/2323db69-fc07-4d2d-bad4-3c7ddce65cd8
      I0313 09:39:40.861481  1734 containerizer.cpp:1598] Checkpointing container's forked pid 1856 to '/tmp/NestedMesosContainerizerTest_ROOT_CGROUPS_LaunchNested_Sf4Iv8/meta/slaves/frameworks/execut[725/1914]or/runs/2323db69-fc07-4d2d-bad4-3c7ddce65cd8/pids/forked.pid'
      I0313 09:39:40.867455  1733 containerizer.cpp:1766] Starting nested container 2323db69-fc07-4d2d-bad4-3c7ddce65cd8.de4bf594-02fe-4ecd-a1b1-6a661b77fe7e
      I0313 09:39:40.870790  1742 linux_launcher.cpp:429] Launching nested container 2323db69-fc07-4d2d-bad4-3c7ddce65cd8.de4bf594-02fe-4ecd-a1b1-6a661b77fe7e and cloning with namespaces CLONE_NEWNS | CLONE_NEW
      PID
      I0313 09:39:45.173310  1737 containerizer.cpp:2483] Container 2323db69-fc07-4d2d-bad4-3c7ddce65cd8.de4bf594-02fe-4ecd-a1b1-6a661b77fe7e has exited
      I0313 09:39:45.173354  1737 containerizer.cpp:2077] Destroying container 2323db69-fc07-4d2d-bad4-3c7ddce65cd8.de4bf594-02fe-4ecd-a1b1-6a661b77fe7e in RUNNING state
      I0313 09:39:45.173630  1731 linux_launcher.cpp:505] Asked to destroy container 2323db69-fc07-4d2d-bad4-3c7ddce65cd8.de4bf594-02fe-4ecd-a1b1-6a661b77fe7e
      I0313 09:39:45.174485  1731 linux_launcher.cpp:548] Using freezer to destroy cgroup mesos_test_03c9934d-4d27-408f-8476-ee1c1983ff1b/2323db69-fc07-4d2d-bad4-3c7ddce65cd8/mesos/de4bf594-02fe-4ecd-a1b1-6a661
      b77fe7e
      I0313 09:39:45.177196  1744 cgroups.cpp:2692] Freezing cgroup /sys/fs/cgroup/freezer/mesos_test_03c9934d-4d27-408f-8476-ee1c1983ff1b/2323db69-fc07-4d2d-bad4-3c7ddce65cd8/mesos/de4bf594-02fe-4ecd-a1b1-6a66
      1b77fe7e
      I0313 09:39:45.179316  1736 cgroups.cpp:1405] Successfully froze cgroup /sys/fs/cgroup/freezer/mesos_test_03c9934d-4d27-408f-8476-ee1c1983ff1b/2323db69-fc07-4d2d-bad4-3c7ddce65cd8/mesos/de4bf594-02fe-4ecd
      -a1b1-6a661b77fe7e after 2.063104ms
      I0313 09:39:45.181565  1733 cgroups.cpp:2710] Thawing cgroup /sys/fs/cgroup/freezer/mesos_test_03c9934d-4d27-408f-8476-ee1c1983ff1b/2323db69-fc07-4d2d-bad4-3c7ddce65cd8/mesos/de4bf594-02fe-4ecd-a1b1-6a661
      b77fe7e
      I0313 09:39:45.183686  1746 cgroups.cpp:1434] Successfully thawed cgroup /sys/fs/cgroup/freezer/mesos_test_03c9934d-4d27-408f-8476-ee1c1983ff1b/2323db69-fc07-4d2d-bad4-3c7ddce65cd8/mesos/de4bf594-02fe-4ec
      d-a1b1-6a661b77fe7e after 2.074112ms
      I0313 09:39:45.187661  1738 containerizer.cpp:2356] Checkpointing termination state to nested container's runtime directory '/tmp/NestedMesosContainerizerTest_ROOT_CGROUPS_LaunchNested_kUGOVz/containers/2
      323db69-fc07-4d2d-bad4-3c7ddce65cd8/containers/de4bf594-02fe-4ecd-a1b1-6a661b77fe7e/termination'
      ../../src/tests/containerizer/nested_mesos_containerizer_tests.cpp:230: Failure
      Expecting WIFEXITED(wait.get()->status()) but  WIFSIGNALED(wait.get()->status()) is true and WTERMSIG(wait.get()->status()) is Killed
      I0313 09:39:45.188985  1737 containerizer.cpp:2077] Destroying container 2323db69-fc07-4d2d-bad4-3c7ddce65cd8 in RUNNING state
      I0313 09:39:45.189237  1734 linux_launcher.cpp:505] Asked to destroy container 2323db69-fc07-4d2d-bad4-3c7ddce65cd8
      I0313 09:39:45.189946  1734 linux_launcher.cpp:548] Using freezer to destroy cgroup mesos_test_03c9934d-4d27-408f-8476-ee1c1983ff1b/2323db69-fc07-4d2d-bad4-3c7ddce65cd8
      I0313 09:39:45.191498  1744 cgroups.cpp:2692] Freezing cgroup /sys/fs/cgroup/freezer/mesos_test_03c9934d-4d27-408f-8476-ee1c1983ff1b/2323db69-fc07-4d2d-bad4-3c7ddce65cd8/mesos
      I0313 09:39:45.191536  1734 cgroups.cpp:2692] Freezing cgroup /sys/fs/cgroup/freezer/mesos_test_03c9934d-4d27-408f-8476-ee1c1983ff1b/2323db69-fc07-4d2d-bad4-3c7ddce65cd8
      I0313 09:39:45.297771  1735 cgroups.cpp:1405] Successfully froze cgroup /sys/fs/cgroup/freezer/mesos_test_03c9934d-4d27-408f-8476-ee1c1983ff1b/2323db69-fc07-4d2d-bad4-3c7ddce65cd8 after 106.187776ms
      I0313 09:39:45.297827  1745 cgroups.cpp:1405] Successfully froze cgroup /sys/fs/cgroup/freezer/mesos_test_03c9934d-4d27-408f-8476-ee1c1983ff1b/2323db69-fc07-4d2d-bad4-3c7ddce65cd8/mesos after 106.283008ms
      I0313 09:39:45.300673  1746 cgroups.cpp:2710] Thawing cgroup /sys/fs/cgroup/freezer/mesos_test_03c9934d-4d27-408f-8476-ee1c1983ff1b/2323db69-fc07-4d2d-bad4-3c7ddce65cd8/mesos
      I0313 09:39:45.301230  1733 cgroups.cpp:2710] Thawing cgroup /sys/fs/cgroup/freezer/mesos_test_03c9934d-4d27-408f-8476-ee1c1983ff1b/2323db69-fc07-4d2d-bad4-3c7ddce65cd8
      I0313 09:39:45.303532  1742 cgroups.cpp:1434] Successfully thawed cgroup /sys/fs/cgroup/freezer/mesos_test_03c9934d-4d27-408f-8476-ee1c1983ff1b/2323db69-fc07-4d2d-bad4-3c7ddce65cd8/mesos after 2.814976ms
      I0313 09:39:45.304054  1741 cgroups.cpp:1434] Successfully thawed cgroup /sys/fs/cgroup/freezer/mesos_test_03c9934d-4d27-408f-8476-ee1c1983ff1b/2323db69-fc07-4d2d-bad4-3c7ddce65cd8 after 2.78272ms
      I0313 09:39:45.374223  1733 containerizer.cpp:2483] Container 2323db69-fc07-4d2d-bad4-3c7ddce65cd8 has exited
      I0313 09:39:45.398809  1731 cgroups.cpp:2692] Freezing cgroup /sys/fs/cgroup/freezer/mesos_test_03c9934d-4d27-408f-8476-ee1c1983ff1b
      I0313 09:39:45.400909  1742 cgroups.cpp:1405] Successfully froze cgroup /sys/fs/cgroup/freezer/mesos_test_03c9934d-4d27-408f-8476-ee1c1983ff1b after 2.054144ms
      I0313 09:39:45.403110  1739 cgroups.cpp:2710] Thawing cgroup /sys/fs/cgroup/freezer/mesos_test_03c9934d-4d27-408f-8476-ee1c1983ff1b
      I0313 09:39:45.405185  1733 cgroups.cpp:1434] Successfully thawed cgroup /sys/fs/cgroup/freezer/mesos_test_03c9934d-4d27-408f-8476-ee1c1983ff1b after 2.034176ms
      [  FAILED  ] NestedMesosContainerizerTest.ROOT_CGROUPS_LaunchNested (4682 ms)
      [----------] 1 test from NestedMesosContainerizerTest (4683 ms total)
      

      Attachments

        Activity

          People

            Unassigned Unassigned
            gkleiman Gastón Kleiman
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: