Continued x86 OpenWRT + Proxmox VLAN issues

I can't get it. I've tried every guide I could find. I did everything I could to not post a question.

  • x86 Intel, 4 ports, switch does not appear in OpenWRT (separate slots as I understand it). 3 NICS in use: WAN, SWITCH, WIFI AP.

  • Asus / FreshTomato in Wireless Access Point mode. Virtual WIFI "GUEST" on tagged with VLAN3

  • ProxMox, 2 vmbr bridges, WAN, SWITCH + WIFI (Port3 + Port4). I created a third "network device" for the OpenWRT VM. vmbr0.3 / VLAN3. This appears "eth2" device in OpenWRT.

  • OpenWRT. Three adapters, wan, eth1, eth2. bl-lan, br-guestwifi. Firewall rules to allow lan to wan, and guestwifi to wan.

  • Private lan is 192.168.1.xxx, works fine. br-gueswifi / eth2 use 192.168.2.1 and SHOULD DHCP serve at 192.168.2.100 - 192.168.2.116

I feel like I'm close!

I can connect to wifi, but instead of being served 192.168.2.1xx ... Guest wifi connections always get 192.168.0.xxx and I have no idea why! I don't see where that is coming from. I didn't want to use DHCP on the ASUS / Tomato because I want visibility at the router. I have visibility at OpenWRT now, just on a range I never allowed or specified.

Any help would be appreciated, but networking is not my first language!

You haven't shared any configs, so nobody can really help at this point.

It is worth stating that running OpenWrt on a VM is not always straight forward as the VM host's networking configuration can be complicated and confusing. If you can run OpenWrt on the bare metal, that's always much easier.

Also, most 4-port NICs for x86 systems are actually 4 individually routed ports, not a switch. They can be bridged together in software, but you won't see a switch menu.

That said, let's start by looking at your config.

Please connect to your OpenWrt device using ssh and copy the output of the following commands and post it here using the "Preformatted text </> " button:
grafik
Remember to redact passwords, MAC addresses and any public IP addresses you may have:

ubus call system board
cat /etc/config/network
cat /etc/config/dhcp
cat /etc/config/firewall

Of course. I wasn't sure what was relevant. Thanks for that list.

ubus

root@router:/# ubus call system board
{
	"kernel": "5.15.134",
	"hostname": "router",
	"system": "Common KVM processor",
	"model": "QEMU Standard PC (i440FX + PIIX, 1996)",
	"board_name": "qemu-standard-pc-i440fx-piix-1996",
	"rootfs_type": "ext4",
	"release": {
		"distribution": "OpenWrt",
		"version": "23.05.0",
		"revision": "r23497-6637af95aa",
		"target": "x86/64",
		"description": "OpenWrt 23.05.0 r23497-6637af95aa"
	}
}

cat /etc/config/network

root@router:/# cat /etc/config/network

config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fdc0:8004:0803::/48'

config device
	option name 'br-lan'
	option type 'bridge'
	list ports 'eth0'

config interface 'lan'
	option device 'br-lan'
	option proto 'static'
	option ipaddr '192.168.1.1'
	option netmask '255.255.0.0'
	option ip6assign '60'

config interface 'wan'
	option proto 'dhcp'
	option device 'eth1'

config device
	option name 'eth1'
	option ipv6 '0'

config device
	option name 'eth0'
	option ipv6 '0'

config device
	option name 'eth2'
	option ipv6 '0'

config device
	option type 'bridge'
	option name 'br-guestwifi'
	option macaddr '9A:0C:A0:E4:BF:BB'
	option ipv6 '0'
	list ports 'eth2'

config interface 'guestwifi'
	option proto 'static'
	option device 'br-guestwifi'
	option ipaddr '192.168.2.1'
	option netmask '255.255.255.0'

cat /etc/config/dhcp

root@router:/# cat /etc/config/dhcp

config dnsmasq
	option domainneeded '1'
	option localise_queries '1'
	option rebind_protection '1'
	option rebind_localhost '1'
	option local '/home.arpa/'
	option domain 'home.arpa'
	option expandhosts '1'
	option cachesize '1000'
	option authoritative '1'
	option readethers '1'
	option leasefile '/tmp/dhcp.leases'
	option resolvfile '/tmp/resolv.conf.d/resolv.conf.auto'
	option localservice '1'
	option ednspacket_max '1232'

config dhcp 'lan'
	option interface 'lan'
	option start '100'
	option limit '99'
	option leasetime '12h'
	option dhcpv4 'server'
	option dhcpv6 'server'
	option ra 'server'
	list ra_flags 'managed-config'
	list ra_flags 'other-config'
	option force '1'

config odhcpd 'odhcpd'
	option maindhcp '0'
	option leasefile '/tmp/hosts/odhcpd'
	option leasetrigger '/usr/sbin/odhcpd-update'
	option loglevel '4'

config dhcp 'wan'
	option interface 'wan'
	option ignore '1'
	option start '100'
	option limit '150'
	option leasetime '12h'

config dhcp 'guestwifi'
	option interface 'guestwifi'
	option start '100'
	option limit '108'
	option leasetime '2h'

cat /etc/config/firewall

root@router:/# cat /etc/config/firewall

config defaults
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option synflood_protect '1'

config zone
	option name 'lan'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'
	list network 'lan'

config zone
	option name 'GWIFI_FW'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'
	list network 'guestwifi'

config zone
	option name 'wan'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option mtu_fix '1'
	option masq '1'
	list network 'wan'

config forwarding
	option src 'lan'
	option dest 'wan'

config rule
	option name 'Allow-DHCP-Renew'
	option src 'wan'
	option proto 'udp'
	option dest_port '68'
	option target 'ACCEPT'
	option family 'ipv4'

config rule
	option name 'Allow-Ping'
	option src 'wan'
	option proto 'icmp'
	option icmp_type 'echo-request'
	option family 'ipv4'
	option target 'ACCEPT'
	option enabled '0'

config rule
	option name 'Allow-IGMP'
	option src 'wan'
	option proto 'igmp'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-DHCPv6'
	option src 'wan'
	option proto 'udp'
	option dest_port '546'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-MLD'
	option src 'wan'
	option proto 'icmp'
	option src_ip 'fe80::/10'
	list icmp_type '130/0'
	list icmp_type '131/0'
	list icmp_type '132/0'
	list icmp_type '143/0'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Input'
	option src 'wan'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	list icmp_type 'router-solicitation'
	list icmp_type 'neighbour-solicitation'
	list icmp_type 'router-advertisement'
	list icmp_type 'neighbour-advertisement'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Forward'
	option src 'wan'
	option dest '*'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-IPSec-ESP'
	option src 'wan'
	option dest 'lan'
	option proto 'esp'
	option target 'ACCEPT'

config rule
	option name 'Allow-ISAKMP'
	option src 'wan'
	option dest 'lan'
	option dest_port '500'
	option proto 'udp'
	option target 'ACCEPT'

config forwarding
	option src 'GWIFI_FW'
	option dest 'wan'

And proxmox for my openwrt VM:

image

And for ProxMox install itself:

That test VLAN3 interface has no effect and is from me testing another method.

The bridges in your network config are not necessary, but they won't hurt anything. Everything on the OpenWrt side is fine.

The issue you are experiencing is related to your proxmox VM environment and/or host os configuration. I can't help there because I don't have the experience with proxmox. Proxmox configuration is bordering on out-of-scope for these forums, but hopefully someone can chime in with a description of how to fix your issues.

1 Like

have you tried pass-through of the nics to openwrt, and manage everything there ?
maybe its easier this way.

as you have one extra port you could use this one for management,
when i started dabbling with proxmox and openwrt thats what i did,
it made the setup less complicated.

i know its not ideal, but maybe you can make it work for now, then later figure it out whats going on.

2 Likes

Pass-Thru NICs was probably my last thing to try. I'm not even sure if this 2014/2015 UEFI bios/board supports it.

It's also basically starting over with ProxMox. I'm not sure how many of my settings will transfer through.

I'm also not sure what that would mean for any other VMs on the machine (I don't currently have any, but I could see a scenario would I might want some).

There are a few arguments against virtualized OpenWrt, so if you don't have any other VMs currently setup or planned, maybe consider running on the bare metal.

The main argument against VMs is simply that you have a host OS and supervisor/hypervisor (or misconfigurations thereof) that could potentially introduce security vulnerabilities or bugs that don't exist within OpenWrt.

Another argument is that your gateway device should not be burdened with other, non-related workloads. This is also relevant from a security standpoint, but also includes processor/resource utilization and maintenance. Even though modern hypvisors/supervisors are quite good at security and stability sandboxing and resource management, negative interactions are still possible. And of course, maintenance of the VM host could require downtime that just wouldn't be necessary with bare-metal configs. And also, you could end up in a chicken-or-egg situation of you encounter a maintenance problem that prevents the system from coming back online properly and that requires internet connectivity to resolve.

There are plenty of valid uses for VMs, too... but if you don't really have a need to run other VMs, it's probably just not worth the hassle to run OpenWrt virtualized.

1 Like

I have uses, but I need this up first.

Replied wrong:

i would suggest you to have another router on hand, so you can try all the configs you want, before losing your connection.

Some time ago i used proxmox with VM home assistant, VM openwrt, LXC adguard.
it worked pretty well but i never used any vlans like you.

i think its a good idea to post the same question on the proxmox forum.

i also don't understand from where you can get this 192.168.0.xxx ip range from, does you wifi AP OS has some type of different behavior for guest wifi that you don't know about, maybe there is some DHCP hidden ?

You initially said that you don't have any other VMs but "could see a scenario where I might want some" -- then you said that you do "have uses." A bit confusing as to how important VMs are in this situation.

It would be good to consider if the various tasks that may run on VMs do actually need to be on the same hardware, and also if any of these potential VMs could actually run inside OpenWrt so you could run bare-metal for OpenWrt itself. For example, there are many packages available for OpenWrt which may cover some of your needs. Additionally, you can actually run Docker on OpenWrt, and that opens a bunch of options, as well.

All of that said, you'll have to experiment with your hypervisor to get the networking to function as desired -- the OpenWrt configuration is fine.... consider that with bare metal you'd already be up and running with OpenWrt (and maybe other things on top of OpenWrt), so keep that in consideration if you continue to have difficulties getting the VM to work properly with the networking stack.

1 Like