I run OpenWRT (openwrt-18.06.2-x86-64-combined-ext4.img) in QEMU on Arch Linux 5.1.2 (exact instructions below) and the virtual tap0 device (in br-lan) cannot reach other subnets than its own. This only happens on OpenWRT (Arch Linux guest is ok).
I use the vanilla stock image mentioned above. The only thing i do is uci set network.lan.ipaddr='192.168.99.1' && uci commit && service network restart
to change the ip. Then going into WebUI setting eth2 to 192.168.2.1/255.255.255.0/<no gateway>/192.168.2.255 (broadcast) and firewall zone to 'eth2'. And lastly, "Network/Firewall" setting FORWARD for 'lan' and 'eth2' zones to ACCEPT.
Now this happens:
On guest side (OpenWRT) you can successfully ping 192.168.2.1, 192.168.99.1 and 192.168.99.99 (the virtual tap0 device from host).
On Host side you can successfully ping 192.168.99.1 (the OpenWRT guest) but not 192.168.2.1 (Destination Host Unreachable). Listening with 'tcpdump -n -i br-lan'
you can see the ICMP requests are arriving at br-lan (who-has 192.168.2.1, tell 192.168.99.99) but OpenWRT is not responding.
- This doesn't happen if you deploy the image to real hardware. I had several LXC containers in subnets that could be accessed without problems from br-lan clients (that's why i need this feature).
- This doesn't happen on Arch Linux guests and tap0 works properly there (it can access other subnets).
The question is: What exactly is preventing the FORWARD for tap0 into other subnets in OpenWRT? There probably is some kind of protection mechanism/option enabled in OpenWRT's kernel that prevents tap0 (or virt devices) to access some areas i think. This might be not a bug but a security feature but i cannot figure out what exactly.
As an ultima ratio i pulled all triggers: Flushed/cleaned all iptables/chains/rules (filter, nat, mangle, raw), all policies to ACCEPT, deleted all interfaces and created just two (eth0, eth1), checked 'ip route' and 'ip neigh' entries. All looks ok, but no change.
I hope for an 'sysctl' or '/proc/sys/net' entry or anyone who knows what's going on here.
Any ideas?
tap0 bridge for QEMU:
#!/bin/bash
if [[ ! $(cat /proc/net/dev |grep br0) ]]; then
echo "Creating bridge"
#Make sure nothing interferes with bridges
#systemctl stop NetworkManager.service
sysctl net.ipv4.ip_forward=1 >/dev/null 2>&1
ip tuntap add tap0 mode tap user 1000
ip tuntap add tap1 mode tap user 1000
ip link set tap0 up
ip link set tap1 up
ip link add dev br0 type bridge
ip link add dev br1 type bridge
ip link set tap0 master br0
ip link set tap1 master br1
ip addr add 192.168.99.99/24 broadcast 192.168.99.255 dev br0
ip addr add 192.168.20.1/24 broadcast 192.168.20.255 dev br1
ip link set dev br0 up
ip link set dev br1 up
ip route add 192.168.2.0/24 dev br0
else
echo "Removing bridge"
ip link set tap0 nomaster
ip link set tap1 nomaster
ip tuntap del tap0 mode tap
ip tuntap del tap1 mode tap
ip addr flush dev br0
ip addr flush dev br1
ip link del dev br0 type bridge
ip link del dev br1 type bridge
sysctl net.ipv4.ip_forward=0 >/dev/null 2>&1
#systemctl start NetworkManager.service
fi
Starting QEMU:
#!/bin/bash
image=/tmp/openwrt-18.06.2-x86-64-combined-ext4.img
# eth2 (socket device) is just used as a dummy here
qemu="qemu-system-x86_64 \
-enable-kvm \
-cpu max \
-smp 4 \
-m 1G \
-boot c \
-drive id=disk0,if=none,format=raw,file=${image} \
-device virtio-blk-pci,drive=disk0 \
-device virtio-net,netdev=tap0 \
-device virtio-net,netdev=tap1 \
-device virtio-net,netdev=eth2 \
-netdev tap,id=tap0,ifname=tap0,script=no,downscript=no \
-netdev tap,id=tap1,ifname=tap1,script=no,downscript=no \
-netdev socket,id=eth2,listen=:33334 \
-serial mon:stdio \
-device qxl-vga \
-device piix3-usb-uhci \
-device usb-tablet \
"
${qemu}
echo -e "\n\033[1;35mBack at host!\033[0m\n"