Installing Docker breaks network connectivity

Installing and Using OpenWrt
1

Immediately after installing dockerd, devices in my home network (brlan.10) become unable talking to each other. They can talk to router (and back), they can access Internet, but cannot connect to/ping each other.

I am guessing that it is Docker's container isolation feature somehow sneaks into the host OS (iptables?). I am not a networking pro, but this seems suspicious to me:

net.bridge.bridge-nf-call-iptables = 1

Are these enabled by default in OpenWrt? Is it compatible with DSA routers?

What else to check?

My device is GL.iNet GL-MT6000.

Configuration 
config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fd6d:67f:d976::/48'
	option packet_steering '1'

config device
	option name 'br-lan'
	option type 'bridge'
	list ports 'lan1'
	list ports 'lan2'
	list ports 'lan3'
	list ports 'lan4'
	list ports 'lan5'

config interface 'lan'
	option device 'br-lan.10'
	option proto 'static'
	option ipaddr '192.168.10.1'
	option netmask '255.255.255.0'
	option ip6assign '60'

config interface 'wan'
	option device 'eth1'
	option proto 'dhcp'

config interface 'wan6'
	option device 'eth1'
	option proto 'dhcpv6'
	option reqprefix '60'
	option reqaddress 'try'
	option norelease '1'

config bridge-vlan
	option device 'br-lan'
	option vlan '10'
	list ports 'lan1'
	list ports 'lan2'
	list ports 'lan3'

config bridge-vlan
	option device 'br-lan'
	option vlan '20'
	list ports 'lan4'
	list ports 'lan5'

config interface 'iot'
	option proto 'static'
	option device 'br-lan.20'
	option ipaddr '192.168.20.1'
	option netmask '255.255.255.0'

config device
	option type 'bridge'
	option name 'br-guest'
	option bridge_empty '1'

config interface 'guest'
	option proto 'static'
	option device 'br-guest'
	option ipaddr '192.168.2.1'
	option netmask '255.255.255.0'

config interface 'docker'
	option device 'docker0'
	option proto 'none'
	option auto '0'

config device
	option type 'bridge'
	option name 'docker0'
-------------------------------------
config defaults
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option synflood_protect '1'

config zone
	option name 'lan'
	list network 'lan'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'

config zone
	option name 'wan'
	list network 'wan'
	list network 'wan6'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'

config forwarding
	option src 'lan'
	option dest 'wan'

config rule
	option name 'Allow-DHCP-Renew'
	option src 'wan'
	option proto 'udp'
	option dest_port '68'
	option target 'ACCEPT'
	option family 'ipv4'

config rule
	option name 'Allow-Ping'
	option src 'wan'
	option proto 'icmp'
	option icmp_type 'echo-request'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-IGMP'
	option src 'wan'
	option proto 'igmp'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-DHCPv6'
	option src 'wan'
	option proto 'udp'
	option dest_port '546'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-MLD'
	option src 'wan'
	option proto 'icmp'
	option src_ip 'fe80::/10'
	list icmp_type '130/0'
	list icmp_type '131/0'
	list icmp_type '132/0'
	list icmp_type '143/0'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Input'
	option src 'wan'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	list icmp_type 'router-solicitation'
	list icmp_type 'neighbour-solicitation'
	list icmp_type 'router-advertisement'
	list icmp_type 'neighbour-advertisement'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Forward'
	option src 'wan'
	option dest '*'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-IPSec-ESP'
	option src 'wan'
	option dest 'lan'
	option proto 'esp'
	option target 'ACCEPT'

config rule
	option name 'Allow-ISAKMP'
	option src 'wan'
	option dest 'lan'
	option dest_port '500'
	option proto 'udp'
	option target 'ACCEPT'

config zone
	option name 'iot'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	list network 'iot'

config forwarding
	option src 'iot'
	option dest 'wan'

config rule
	option src 'iot'
	option name 'Allow-IoT-DHCP'
	list proto 'udp'
	option dest_port '67-68'
	option target 'ACCEPT'

config rule
	option src 'iot'
	option name 'Allow-IoT-DNS'
	option dest_port '53'
	option target 'ACCEPT'

config zone
	option name 'guest'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	list network 'guest'

config forwarding
	option src 'guest'
	option dest 'wan'

config rule
	option src 'guest'
	option name 'Allow-guest-DHCP'
	list proto 'udp'
	option dest_port '67-68'
	option target 'ACCEPT'

config rule
	option src 'guest'
	option name 'Allow-guest-DNS'
	option dest_port '53'
	option target 'ACCEPT'

config zone 'docker'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'
	option name 'docker'
	list network 'docker'
2

What version of OpenWrt are you running?

1 Like
3

Read here:

1 Like
4

Thanks a lot @Hudra. I think this must be in the guide...

1 Like
5

Why are you installing docker in your router?
Do you have a significant reason to do this? Because I highly advise against this. Routers are definitely not suitable machines to run any form of virtualization, beside the glaring security issues, you also have performance issues.
You also appear to be using your router as a file server (correct me if I am wrong)?
If you want to be serious in your networking, consider the "1 server, 1 usecase" principle. This is especially true for your router, let it do router things such and vlan gateway, routing and firewalling. Keep all your other services hosted on a VM server and then decide if the router should port forward these services or not

2 Likes
6

Your recommendation is understood and appreciated. In the ideal world you are absolutely right, and there should be a separate host for Docker workloads, but I am not an enterprise, and in my home network there are just two devices that are always on: router and NAS. NAS is on a pretty weak hardware platform and cannot run Docker (officially), so the router remains the only choice, unless I invest a couple of hundred dollars into another fanless computer.

Not planning to use it as a file server (I have NAS already). If you are referring to my recent questions related to SMB, those were about client capability (to access that NAS). Also Docker is not really a virtualization system (it's process/network isolation) and has pretty manageable performance overhead. So it's rather a function of what we run in those containers to make the performance or other impact.

One use case I have is running a torrent client (like this). The container would utilize the CIFS mount to read/save torrent data on NAS. This one could have the biggest performance impact potentially, other use cases are really miniscule web clients/servers that I don't think will be even noticed in the system.

I wonder, which glaring security issues are you referring to? Native Docker isolation rather gives much better levels of security and control than running workloads natively in the OS. Networking-wise I am not going to do any compromises, e.g. running in host mode, disabling isolation, etc. All container access will be firewalled as any other traffic, with only required access open.

So my plan, I guess, is to try and see how it works, watch CPU/networking/memory charts, etc. In case of unresolvable troubles I always can remove Docker and switch to an alternative.

7

My advice in a situation like yours is simple. Get an old or refurbished x86-64 machine, use debian as your OS, and use this machine as your personal server / docker host. Energy wise, they can get pretty efficient for 24/7 operation and you end up saving a lot of money that way, while still following the 1 server, 1 usecase principle.

For starters, openwrt has only one user "root". Running docker as root is a know and serious security risk. openwrt was never designed to run docker containers, you need to do some serious hardening in order to ensure a "safe" (emphasis on the "") operation.

Security vulnerabilities explore the craziest, most unimaginable attack vectors, and adding docker to a system adds a giant attack vector, further compounded by the containers running. Adding all this to a system that's the most exposed to attacks, is just asking for trouble IMO.

1 Like