I checked multiple old threads regarding docker setup with nftables as there's no official support or finished migration from docker side. However from forum discussions it seems there's a way to properly setup docker w/o iptables
Main problem with docker setup in the current state is the bridge mode type of network.
What I did so far:
Created separate network and use it as default network for containers: docker network create -o com.docker.network.bridge.enable_icc=true -o com.docker.network.bridge.enable_ip_masquerade=true dockerlan -o com.docker.network.bridge.host_binding_ipv4=0.0.0.0 -o com.docker.network.bridge.name=docker1 --ip-range=172.19.0.0/24 --subnet 172.19.0.0/27 --gateway=172.19.0.1
added following firewall rules
config zone 'docker'
option input 'ACCEPT'
option output 'ACCEPT'
option name 'docker'
option forward 'ACCEPT'
option log '1'
list network 'dockerlan'
list network 'docker'
list device 'docker0'
list device 'docker1'
config rule
option name 'DockertoDockerAny'
option src 'docker'
option dest 'docker'
option target 'ACCEPT'
config forwarding
option src 'docker'
option dest 'wan'
dockerd config:
cat /etc/config/dockerd
# The following settings require a restart of docker to take full effect, A reload will only have partial or no effect:
# log_driver
# bip
# blocked_interfaces
# extra_iptables_args
# device
config globals 'globals'
# option alt_config_file '/etc/docker/daemon.json'
option data_root '/opt/docker/'
# option log_driver 'local'
option log_level 'warn'
option iptables '0'
# list hosts 'unix:///var/run/docker.sock'
# option bip '172.18.0.1/24'
# option fixed_cidr '172.17.0.0/16'
# option fixed_cidr_v6 'fc00:1::/80'
# option ipv6 '1'
# option ip '::ffff:0.0.0.0'
# list dns '172.17.0.1'
# list registry_mirrors 'https://<my-docker-mirror-host>'
# list registry_mirrors 'https://hub.docker.com'
# Docker doesn't work well out of the box with fw4. This is because Docker relies on a compatibility layer that
# naively translates iptables rules. For the best compatibility replace the following dependencies:
# `firewall4` -> `firewall`
# `iptables-nft` -> `iptables-legacy`
# `ip6tables-nft` -> `ip6tables-legacy`
# Docker undermines the fw3 rules. By default all external source IPs are allowed to connect to the Docker host.
# See https://docs.docker.com/network/iptables/ for more details.
# firewall config changes are only additive i.e firewall will need to be restarted first to clear old changes,
# then docker restarted to load in new changes.
config firewall 'firewall'
option device 'docker0'
list blocked_interfaces 'wan'
# option extra_iptables_args '--match conntrack ! --ctstate RELATED,ESTABLISHED' # allow outbound connections
I can access containers from local network, but there's no internet connection with network created and no connection between containers or to the host itself.
As there's no straightforward steps and every discussion troubleshoots different cases. Can we have discussion dedicated to the proper setup and steps to be done prior docker/after docker is installed?
On the same note,
I'd love to know how to completely disable all default docker networks and remove attached firewall settings.
As I prefer to use a macvlan network instead for all my containers.
I prefer it this way because each container gets its own IP and generally its just like if I would plug a new device in to my network, well except for docker only supporting its internal DHCP, but I can live with setting the IP in the compose file.
For the curios, my current setup:
(I'd love to add it to the official container guide, but need help translating it in to UCI commands)
To create the mac-vlan network edit /etc/config/network and add this section to the bottom:
config interface 'vlan20' ## name however you want
option device 'br-lan.20'
option proto 'static'
option ipaddr '192.168.20.1'
option netmask '255.255.255.0'
option ip6assign '80' # docker recommends to use a \80 network
option ip6hint '20'
config device
option type 'macvlan'
option ifname 'br-lan'
option mode 'bridge'
option name 'br-lan.20'
option acceptlocal '1'
Add the following to your docker-compose.yml I prefer the top just under version:
And this to the network section of each container / services:
networks: # defines the network(s) this container is connected to
lan20:
ipv4_address: 192.168.20.20 # defines the IPv4 address, the container will use
ipv6_address: fd19:7219:b304:20::20 # local private IPv6 address no internet access only our internal network
# ipv6_address: 2406:2d40:7238:9a20::20 # public IPv6 address fully functional IPv6 internet acces
The common problem with containers not communicating with each other over a user bridge network (typical when using docker compose) is that upon creating the bridge, Docker constructs the interface but does not add it to the Docker zone in the fw4 firewall. The solution was to find the bridge interface id (eg. br-a7cece5acbfe) then add it to the Docker firewall zone, either through Luci or via uci from the command line, then restart the firewall.
Through Luci, the steps are:
Go to Network->Firewall
Zones->docker , click Edit
Advanced Settings tab, add/select the device id under "Covered Devices" - > Save
Save & Apply.
UCI through the command line:
brctl show
uci add_list firewall.docker.device=bridge_interface_id_here
uci commit firewall
service firewall reload
where bridge_interface_id_here = the docker interface id (eg br-a7cece5acbfe )
if the Docker bridge network is removed, it should also be cleared from the firewall zone with
uci del_list firewall.docker.device=bridge_interface_id_here
uci commit firewall
service firewall reload