Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
Fedora 23 with network isolation
-
Mesosphere Sprint 37
-
3
Description
Port mapping isolator may return failure in isolate method, if a symlink to the network namespace handle using that ContainerId already existed. We should overwrite the symlink if it exist.
This affects a couple test failures:
PortMappingIsolatorTest.ROOT_TooManyContainers PortMappingIsolatorTest.ROOT_ContainerARPExternal PortMappingIsolatorTest.ROOT_ContainerCMPInternal PortMappingIsolatorTest.ROOT_NC_HostToContainerTCP
Here is an example failure test log:
[00:28:37] : [Step 10/10] [ RUN ] PortMappingIsolatorTest.ROOT_TooManyContainers
[00:28:37]W: [Step 10/10] I0606 00:28:37.046444 24846 port_mapping_tests.cpp:229] Using eth0 as the public interface
[00:28:37]W: [Step 10/10] I0606 00:28:37.046728 24846 port_mapping_tests.cpp:237] Using lo as the loopback interface
[00:28:37]W: [Step 10/10] I0606 00:28:37.058758 24846 resources.cpp:572] Parsing resources as JSON failed: cpus:2;mem:1024;disk:1024;ephemeral_ports:[30001-30999];ports:[31000-32000]
[00:28:37]W: [Step 10/10] Trying semicolon-delimited string format instead
[00:28:37]W: [Step 10/10] I0606 00:28:37.059711 24846 port_mapping.cpp:1557] Using eth0 as the public interface
[00:28:37]W: [Step 10/10] I0606 00:28:37.059998 24846 port_mapping.cpp:1582] Using lo as the loopback interface
[00:28:37]W: [Step 10/10] I0606 00:28:37.061126 24846 port_mapping.cpp:1869] /proc/sys/net/ipv4/neigh/default/gc_thresh3 = '1024'
[00:28:37]W: [Step 10/10] I0606 00:28:37.061172 24846 port_mapping.cpp:1869] /proc/sys/net/ipv4/neigh/default/gc_thresh1 = '128'
[00:28:37]W: [Step 10/10] I0606 00:28:37.061206 24846 port_mapping.cpp:1869] /proc/sys/net/ipv4/tcp_wmem = '4096 16384 4194304'
[00:28:37]W: [Step 10/10] I0606 00:28:37.061256 24846 port_mapping.cpp:1869] /proc/sys/net/ipv4/tcp_synack_retries = '5'
[00:28:37]W: [Step 10/10] I0606 00:28:37.061297 24846 port_mapping.cpp:1869] /proc/sys/net/core/rmem_max = '212992'
[00:28:37]W: [Step 10/10] I0606 00:28:37.061331 24846 port_mapping.cpp:1869] /proc/sys/net/core/somaxconn = '128'
[00:28:37]W: [Step 10/10] I0606 00:28:37.061360 24846 port_mapping.cpp:1869] /proc/sys/net/core/wmem_max = '212992'
[00:28:37]W: [Step 10/10] I0606 00:28:37.061390 24846 port_mapping.cpp:1869] /proc/sys/net/ipv4/tcp_rmem = '4096 87380 6291456'
[00:28:37]W: [Step 10/10] I0606 00:28:37.061419 24846 port_mapping.cpp:1869] /proc/sys/net/ipv4/tcp_keepalive_time = '7200'
[00:28:37]W: [Step 10/10] I0606 00:28:37.061450 24846 port_mapping.cpp:1869] /proc/sys/net/ipv4/neigh/default/gc_thresh2 = '512'
[00:28:37]W: [Step 10/10] I0606 00:28:37.061480 24846 port_mapping.cpp:1869] /proc/sys/net/core/netdev_max_backlog = '1000'
[00:28:37]W: [Step 10/10] I0606 00:28:37.061511 24846 port_mapping.cpp:1869] /proc/sys/net/ipv4/tcp_keepalive_intvl = '75'
[00:28:37]W: [Step 10/10] I0606 00:28:37.061540 24846 port_mapping.cpp:1869] /proc/sys/net/ipv4/tcp_keepalive_probes = '9'
[00:28:37]W: [Step 10/10] I0606 00:28:37.061569 24846 port_mapping.cpp:1869] /proc/sys/net/ipv4/tcp_max_syn_backlog = '512'
[00:28:37]W: [Step 10/10] I0606 00:28:37.061599 24846 port_mapping.cpp:1869] /proc/sys/net/ipv4/tcp_retries2 = '15'
[00:28:37]W: [Step 10/10] I0606 00:28:37.069964 24846 linux_launcher.cpp:101] Using /sys/fs/cgroup/freezer as the freezer hierarchy for the Linux launcher
[00:28:37]W: [Step 10/10] I0606 00:28:37.070144 24846 resources.cpp:572] Parsing resources as JSON failed: ports:[31000-31499]
[00:28:37]W: [Step 10/10] Trying semicolon-delimited string format instead
[00:28:37]W: [Step 10/10] I0606 00:28:37.070677 24867 port_mapping.cpp:2512] Using non-ephemeral ports {[31000,31500)} and ephemeral ports [30208,30720) for container container1 of executor ''
[00:28:37]W: [Step 10/10] I0606 00:28:37.071688 24846 linux_launcher.cpp:281] Cloning child process with flags = CLONE_NEWNS | CLONE_NEWNET
[00:28:37]W: [Step 10/10] I0606 00:28:37.084079 24863 port_mapping.cpp:2576] Bind mounted '/proc/11997/ns/net' to '/run/netns/11997' for container container1
[00:28:37] : [Step 10/10] ../../src/tests/containerizer/port_mapping_tests.cpp:1438: Failure
[00:28:37] : [Step 10/10] (isolator.get()->isolate(containerId1, pid.get())).failure(): Failed to symlink the network namespace handle '/var/run/mesos/netns/container1' -> '/run/netns/11997': File exists
[00:28:37] : [Step 10/10] [ FAILED ] PortMappingIsolatorTest.ROOT_TooManyContainers (57 ms)