Hi all!
I am sorry for this very long post, but I wanted to detail everything in order to help someone who can benefit for a setup similar to mine.
After long thinkering time, I decided to put some effort into try to have a better control over my network using OpenWRT, but since I don't have an available router to flash, I am stuck with my laptop that have to run Windows due work reasons.
I use QEMU for the virtualization because I want to learn that tool instead of using high-level solution like VirtualBox or VMWare, it should be sufficient for my case.
The goal is letting an OpenWRT Virtual Machine to manage the routing/firewalling of my network, and force devices that connect to the WiFi of my ISP router to take the route of the OpenWRT VM, which is quite easyly done by using DHCP with the default gateway to OpenWRT IP address.
To achieve this, I understood I need two different "ports" (e.g. network card) for OpenWRT in order to route LAN and WAN traffic.
My very artistical representation of the setup I managed to get to work (at least, that's how I understood it)
___________
|eth0 (LAN)| <-->tap0<--> Bridge <--> eth (windows) <--> ISP router (WiFi)
| | ^
|eth1 (WAN)| <------>qemu-user<-------------|
___________
OpenWRT VM
I know this isn't quite right, since qemu-user does not work like that, but I hope the point is clear: the WAN port must pass through host interface (cause in my physical PC i have only that one).
The network addresses are:
- LAN: 192.168.0.0/24
- WAN: 192.168.100.0/24
- ISP: 192.168.0.254, 192.168.100.254 (it is possible to have double addresses)
- tap0+eth windows (since it's a bridged connection, it has unique IP): 192.168.0.10
- eth0 (OpenWRT LAN address): 192.168.0.1
- eth1 (OpenWRT WAN address): 192.168.100.15
The steps to reproduce this solution has a prerequisite: one has to create the tap0 device using the OpenVPN utility tapctl.exe create --name tap0 (or any other way to create a tap device), and then bridge the connection between tap0 and eth (windows) in the Network Connections panel. In my system, I had some trouble to get it to work: I had to fire up the QEMU machine (so the link becomes ready), create the bridge, shut down the QEMU machine, restart the QEMU machine (not a reboot!) and everything after that was fine. Of course, I changed the OpenWRT LAN address using `uci set network.lan.ipaddr='192.168.0.1'; uci commit; service network restart .
The command to fire the machine is (the caret is the Windows equivalent of Linux backslash for multiline commands)
qemu-system-x86_64.exe ^
-accel whpx ^
-M q35 ^
-drive file=openwrt-19.07.4-x86-64-combined-ext4.img,id=d0,if=none,bus=0,unit=0 ^
-device ide-hd,drive=d0,bus=ide.0 ^
-netdev tap,id=lan,ifname=tap0,script=no,downscript=no,^
-device virtio-net-pci,netdev=lan,mac=52:54:98:76:54:32^
-netdev user,id=wan,net=192.168.100.0/24 ^
-device virtio-net-pci,netdev=wan,mac=52:54:98:76:54:33
While this setup "just works" (it may be enough for users which needs to get the things done using Windows and QEMU, like the guide says), I don't like to have the qemu-user network as WAN bridge, since it's enough for normal stuffs but things like ping does not work. Moreover, it adds too much things (a firewall, a DHCP server, ...) which does not reflect a true OpenWRT setup on a real machine
Just to recap how QEMU user network emulation works, it basically create a separate network and route the traffic into existing environment.
I managed to get that working using only one card in OpenWRT (ditching the "wan" device) and by adding a route through ip route add 192.168.100.0/24 dev br-lan src 192.168.0.1 (i tried using via instead of src and that wasn't working).
I still have not clear the concept on how the same device has two different IP address: the bridged interface in Windows have 192.168.0.10 (so, same IP address for two cards that has been logically connected as one, as far as I understood) while inside of OpenWRT I have another card with another MAC address and its own IP.
My understanding is that the tap0 device is needed by QEMU to "talk" with Windows networking, and internally Windows handles the addressing translating the ethernet requests from tap0 to the bridge, basically acting like a switch (arp -a shows the MAC address of QEMU). Is this statement correct?
However, finally I was able to perform the setup I needed: I modified the wan interface to be an alias of lan (even if I think it's bad practice), set it to the static address 192.168.100.1 with default gateway to 192.168.0.254 (i.e., the ISP router) and everything is working as expected: a device with default gateway the LAN address of OpenWRT was able to navigate to the internet, and activating a firewall rule that blocks the traffic from lan to wan, the same device cannot reach the outside.
To summarize everything, i have a level 4 separation, but neither 3 or 2, since OpenWRT just point at another device in the same network, but it is seems ok for my scope, which is basically throttling download speed. I tested with the luci-app-nft-qos package and it works beautifully, both in download and upload.
If I understand correctly the way it works, the steps that need to be done to transform this configuration into a "true" OpenWRT router are:
- Put the ISP router in a different network, and let OpenWRT manages all LAN-related stuff, to provide level 3 isolation
- Use VLAN (or different network card) to provide level 2 separation
Due having configuration segmented in the logical card lan and wan, it should be possible just to swap the "wan" configuration to a true card and every defined rule that uses wan should reflect the changes and work flawlessly, is my understanding correct? It would be lovely, so I am able to test OpenWRT using my machine, while maintaing every other device in the network connected to the Internet using another gateway.
The final routing is something like this
Phone: 192.168.0.20 -> OpenWRT LAN 192.168.0.1 -> OpenWRT WAN 192.168.100.1 ->
ISP router 192.168.0.1 -> Internet!
Due the bridging between tap0 and eth (windows), the VM acts like a physical machine in the network, so I didn't have to mess with routing table, so no external configuration is necessary
For reference, this is the network configuration for an OpenWRT router with LAN address 192.168.0.1 and gateway for the internet on 192.168.0.254
/etc/config/network
config interface 'loopback'
option ifname 'lo'
option proto 'static'
option ipaddr '127.0.0.1'
option netmask '255.0.0.0'
config globals 'globals'
option ula_prefix 'fd77:1999:0d30::/48'
config interface 'lan'
option type 'bridge'
option ifname 'eth0'
option proto 'static'
option netmask '255.255.255.0'
option ip6assign '60'
option ipaddr '192.168.0.1'
config interface 'wan'
option ipaddr '192.168.100.1'
option netmask '255.255.255.0'
option proto 'static'
option ifname '@lan'
option gateway '192.168.0.254'
list dns '192.168.0.254'
launch_qemu.bat
C:\"Program Files"\qemu\qemu-system-x86_64.exe ^
-accel whpx ^
-M q35 ^
-drive file=openwrt-19.07.4-x86-64-combined-ext4.img,id=d0,if=none,bus=0,unit=0 ^
-device ide-hd,drive=d0,bus=ide.0 ^
-netdev tap,id=lan,ifname=tap0,script=no,downscript=no,^
-device virtio-net-pci,netdev=lan,mac=52:54:98:76:54:32
So, to recap the steps needed to have this setup working in my configuration (which, for reference, can be schematized like this, all connected via ethernet):
PC <-> Wireless AP 1 <-> Wireless AP 2 <-> Router ISP <-> Internet
Addresses
PC: 192.168.0.10
Wireless AP 1: 192.168.0.253
Wireless AP 2: 192.168.0.254, 192.168.1.254
Router ISP: 192.168.1.1
This configuration is a little more complicated than the one presented, but it may be beneficial to understand why the wireless part is not working, maybe is due to layer 2 problems. I doubt it, tho
- Create TAP interface using OpenVPN utility
tapctl.exe create --name tap0. You can obtain it by intalling OpenVPN GUI and then you find the binary in "C:\Program Files\OpenVPN\bin" - Execute the
launch_qemu.bat - While the OpenWRT machine is powered up, the "tap0" interface should indicate "Connected". The command to open this window is
ncpa.cpl - Select the
tap1interface and the connection you use to connect to the network. I tested the ethernet one, wireless seems more problematic. I tried that too, but the gateway instantly resolve to the "real" gateway. I am not interested in that solution, so I decided to not pursue further investigation. - In the QEMU window, adjust
/etc/config/networkwith the file above (or your modifications to it) - Issue the
poweroffcommand for QEMU - Re-lauinch
launch_qemu.bat(it seems to be something weird about Windows driver which puttap0into disconnected state, reboot solve that issue) - Now if a device has default gateway of 192.168.0.1, then it is routed through OpenWRT!
- Drink a beer, you deserve it
One final remarks: I was forced to do this setup because I wasn't able to connect two tap devices using QEMU.I tought it would be sufficient to create a tap1 interface in Windows, grouping it with the eth (Windows) and tap0, and assign it to the VM. In this way, OpenWRT should see 2 physical devices, which are both connected to the real ethernet card using Windows as a switch, and thus everything should be plug and play.
However, there are some problems in QEMU, since when I try to run the QEMU machine with the second tap device (created with tapctl.exe create --name tap1) with the options for QEMU -netdev tap,id=lan,ifname=tap1,script=no,downscript=no, -device virtio-net-pci,netdev=lan,mac=52:54:98:76:54:33, I am not able even to reach the GRUB selection between OpenWRT and its failsafe variant, so it seems it's something related to QEMU configuration.If someone has some tips to help debugging, they are all welcome, since I have no idea on how to investigate the problem.