MACVLAN devices are not seen and picked up by the firewall rules

Installing and Using OpenWrt
1

Hi all,

I am having a firewall challenge when using macvlan networks in docker. I hope you can help with your experience of how to overcome this challenge.

Environment:

PC x86/64 running OpenWrt 23.05.2 with a single 1Gb network port connected to VLAN capable managed switch.

OpenWrt configuration:

  • WAN interface on device eth0.2 vid 2
  • LAN interface on device eth0.10 vid 10
  • LAB interface on device eth0.20 vid 20
  • Firewall zone LAN forwarding to zone LAB & WAN and from LAB accept, accept, accept (covered networks LAN)
  • Firewall zone LAB forwarding to zone LAN & WAN and from LAN accept, accept, reject (covered networks LAB)

What is working (as expected):

  • All hosts on LAB are reachable from hosts on LAN and from openwrt sh (ping, ssh, etc.)
  • All hosts on LAN are reachable from hosts on LAB and from openwrt sh (ping, ssh, etc.)
  • Both LAN and LAB can reach the internet through WAN.

The challenge I am having:

Installed docker on openwrt with opkg (v24.0.5 and compose v2.18.1, no issues)
Installed kmod-macvlan (no issues)

# docker network create -d macvlan --subnet 192.168.10.0/24 --gateway=192.168.10.1 --ip-range 192.168.10.253/32 -o parent=eth0.10 lan (no issues)
# docker run -itd --rm --network lan --ip 192.168.10.125 --name c1 nginx:alpine (no issues)

  • All hosts on LAN can reach NGINX webserver c1 (good)
  • Openwrt sh cannot ping NGINX webserver c1 (not expected)
  • All hosts on LAB cannot reach NGINX webserver c1 (not expected)

# docker network create -d macvlan --subnet 192.168.20.0/24 --gateway=192.168.20.1 --ip-range 192.168.20.253/32 -o parent=eth0.20 lab (no issues)
# docker run -itd --rm --network lab --ip 192.168.20.126 --name c2 nginx:alpine (no issues)

  • All hosts on LAN cannot reach NGINX webserver c2 (not expected)
  • Openwrt sh cannot ping NGINX webserver c2 (not expected)
  • All hosts on LAB can reach NGINX webserver c2 (good)
  • # docker exec -it c1 sh cannot ping c2 (not expected)
  • # docker exec -it c2 sh cannot ping c1 (not expected)

Conclusion so far:

The macvlan devices are working properly and are connected to their vlan on eth0.

The macvlan devices are not seen and picked up by the firewall rules that UCI manages.

Am I correct in this assessment?

And what configuration changes need to be made to the firewall rules to correct this?

2

This is expected. The host with the macvlan network cannot reach the docker container, AFAIK this is a security measure implied by the Linux kernel. You can work around this by creating a shim network interface. Please use a search engine for this.

I cannot comment on your other questions as I don't run docker on OpenWrt but on a separate machine.

3

Thank you for your review and response. Fortunately I am less concerned about openwrt sh or the two containers not being able to communicate in my particular case.
The fact that the openwrt firewall does not forward traffic from one vlan to the other vlan where the macvlan container resides is more a concern for me.
Hopefully others with a similar need have solved it will share their solution.