Gretap/vxlan over WAN port (Archer A7) with vlans

robca · December 13, 2024, 6:57pm

I have a 3 router setup in my place: a main OpenWRT 23.05 router interfaced to the internet and providing all the main functionality (firewall, VPN, adblock, DNS, DHCP, etc), plus a couple of OpenWRT routers configured as APs, each providing a series of SSIDs and sending traffic to the main router.

My network has 3 VLANs, one for full access, one limited for IoT devices (only internet access) one even more limited for Chinese security cameras (no internet access, can't connect to the main network, but the main netrowk devices can see the cameras)

One of the APs is in a detached garage, connected with a 2.4GHz bridge (TP-Link CPE210). That bridge is vlan-aware and the traffic travels with no problems. But the link is unreliable due to vegetation and weather interfering.

I want to use a 802.11ah (halow) 900MHz bridge instead. Slower, but much more reliable. Alas, as far as I can tell, 802.11ah doesn't support 802.1Q headers properly, and the vlan packets become unreliable (hard to pinpoint what's happening: some traffic goes thru, other doesn't)

Is there a way to configure the AP in the garage to somehow "tunnel" its traffic like a VPN, so that the packets traveling across the bridge are just plain packets with no vlan headers, and on the receiving end have the main router "unpack" the traffic? Apologies for not using the proper terminology, I hope it makes sense. Basically some sort of "internal VPN" on the traffic between an AP and the main router, which are connected by a non-vlan-aware bridge

brada4 · December 13, 2024, 7:45pm

Anyone on untagged, like connected to physical port, will be able to access all VLAN-s, and some not-smart switches do not forward bigger frames. You have to check that first.

robca · December 13, 2024, 7:55pm

The current network functionality is fine, and I'm satisfied with the current level of security (nobody can connect on an untagged port). I used this configuration for more than 2 years with no problems (aside from the CPE210 glitches due to vegetation/weather)

What I need is the ability to bridge traffic over a 802.11ah bridge that doesn't support 802.1Q vlan tagging

brada4 · December 13, 2024, 7:59pm

wifi does not support vlan tagging, you have to increase wifi interface MTU and pass gretap or gre tunnel across.

robca · December 13, 2024, 11:09pm

Thanks for pointing me in the gretap direction, super helpful! There are actually commercial wireless bridges that support 802.1Q-style vlan-tagged packets over wireless (my CPE210 is an example), but those are rare.

What is the most lightweight tunneling protocol available in OpenWRT? https://openwrt.org/docs/guide-user/network/tunneling_interface_protocols 802.11ah is slow enough, so reducing overhead would be nice

I see multiple forum posts about using OpenWRT devices to create a wireless trunk, but in my case I'm using dedicated hardware so, from the OpenWRT router and AP point of view, the devices are connected by a wire. To add complexity, due to the house wiring, the wireless bridge receiver on the router side goes thru a switch, but the tunnel IP address hopefully should take care of it.

So I'm trying to adapt this https://github.com/oofnikj/openwrt-gretap?tab=readme-ov-file to use the backhaul between AP and router instead of the wireless network "trunk" in that example (and probably getting lost in the unfamiliar tunnel terminology).... need to do more reading, suggestions welcome

brada4 · December 13, 2024, 11:11pm

gretap as i already said? Yo need to add overhead to wireless mtu, ymmv with proprietary drivers.

robca · December 16, 2024, 1:07am

Thanks for confirming it. gretap it is. I installed the gre package (and all its dependencies)

But I'm still lost, sorry. I started from a clean configuration on my Archer A7 as "dumb AP". I disabled DHCP, defined a main vlan (1) and two additional vlans, 101 and 102. Port LAN 1 s vlan 101, port 2 is vlan 102. At the moment the WAN port is not used.

I'd like to have all the lan/vlans traffic tunneled by gretap over the WAN port (blue connector). Here is my network setup


config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fd87:39a5:056c::/48'

config device
	option name 'br-lan'
	option type 'bridge'
	list ports 'eth0.1'

config interface 'lan'
	option device 'br-lan'
	option proto 'static'
	option ipaddr '192.168.1.6'
	option netmask '255.255.255.0'
	option ip6assign '60'
	option gateway '192.168.1.1'
	list dns '192.168.1.1'

config switch
	option name 'switch0'
	option reset '1'
	option enable_vlan '1'

config switch_vlan
	option device 'switch0'
	option vlan '1'
	option ports '0t 4 5 1t'
	option vid '1'

config switch_vlan
	option device 'switch0'
	option vlan '2'
	option ports '0t 2 1t'
	option vid '101'

config switch_vlan
	option device 'switch0'
	option vlan '3'
	option ports '0t 3 1t'
	option vid '102'

config device
	option name 'br-iot'
	option type 'bridge'
	list ports 'eth0.101'

config device
	option name 'br-camera'
	option type 'bridge'
	list ports 'eth0.102'

config interface 'iot'
	option device 'br-iot'
	option proto 'none'

config interface 'camera'
	option device 'br-camera'
	option proto 'none'

Can someone please point me in the right direction? All the examples I find are using wifi as the backhaul, I want to use the WAN port instead (eth0.2 in my router)

greybeard · December 16, 2024, 2:26am

vxlan works as well. If you want to use WAN port instead of wifi as the trunk then substitute the wan link for the wifi link. You potentially have an MTU issue though as you have no control over the WAN mtu size, but adjusting the mtu on the vxlan (or gre) link will sort that out.
Same deal if you want to tunnel it through a vpn.

robca · December 17, 2024, 11:15pm

Thanks! Let's stick to greatap, which I understand slightly better than vxlan (at least in theory, as you can see I'm still lost in practice)

Here's my network config in the AP router (Archer A7). I haven't started adding anything yet to the main router, as I want to make sure I'm at least pointed in the right direction before making the main router unreliable


config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fd87:39a5:056c::/48'

config device
	option name 'br-lan'
	option type 'bridge'
	list ports 'eth0.1'
	list ports '@trunk.1'

config interface 'lan'
	option device 'br-lan'
	option proto 'static'
	option ipaddr '192.168.1.6'
	option netmask '255.255.255.0'
	option ip6assign '60'
	option gateway '192.168.1.1'
	list dns '192.168.1.1'

config switch
	option name 'switch0'
	option reset '1'
	option enable_vlan '1'

config switch_vlan
	option device 'switch0'
	option vlan '1'
	option ports '0t 4 5 1t'
	option vid '1'

config switch_vlan
	option device 'switch0'
	option vlan '2'
	option ports '0t 2 1t'
	option vid '101'

config switch_vlan
	option device 'switch0'
	option vlan '3'
	option ports '0t 3 1t'
	option vid '102'

config device
	option name 'br-iot'
	option type 'bridge'
	list ports 'eth0.101'
	list ports '@trunk.101'

config device
	option name 'br-camera'
	option type 'bridge'
	list ports 'eth0.102'
	list ports '@trunk.102'

config interface 'iot'
	option device 'br-iot'
	option proto 'none'

config interface 'camera'
	option device 'br-camera'
	option proto 'none'

config interface 'gre'
	option proto 'gretap'
	option ipaddr '10.99.0.2'
	option peeraddr '10.99.0.1'
	option tunlink 'wtun'
	option network 'trunk'
	option df '0'
	option mtu '1500'

config interface 'wtun'
	option proto 'static'
	option ipaddr '10.99.0.2'
	option netmask '255.255.255.0'
	option delegate '0'
	option mtu '2048'

config interface 'trunk'
	option proto 'none'
	option auto '1'
	option bridge_empty '1'
	option delegate '0'
	option stp '1'
	option device 'br-trunk'

config device
	option name 'br-trunk'
	option type 'bridge'
	list ports 'eth0.2'
	option igmp_snooping '1'

Am I close to have a working configuration? In my router, port 0 of the switch is the CPU, port 1 is the WAN port (blue), port 2-5 are the 4 yellow LAN ports. eth0.1 is the 4 yellow lan ports, eth0.2 the blue WAN port

greybeard · December 18, 2024, 5:37am

I don't use gre however I adapted a gre setup for my vxlan config.
Just take basic steps first.
Setup a link between the two routers. just the link, don't connect it to any other interface/networks.
From the console (ssh) make sure you have connectivity between the devices on your new link, ping and then traceroute to make sure you are going via your link.
Add the grep layer over the new link and bridge it at either end to a new interface (so you can assign an IP address), ping and traceroute as before.
Once you have that sorted, start bridging vlan's onto the link and repeat your ping/traceroute test.
Keep it simple to start with, make sure you use unique networks/address ranges and test and confirm at each stage.
So, link, tunnel, networks, beer o'clock!

robca · December 19, 2024, 12:42am

That makes a lot of sense, thanks!

I guess I got stuck right away, unfortunately.

With this network configuration file I can connect the Archer A7 to my main router, and it all works. The Archer is connected via the blue WAN port, which is switched as port 1 (and part of the vlans)

config device
	option name 'br-lan'
	option type 'bridge'
	list ports 'eth0.1'

config interface 'lan'
	option device 'br-lan'
	option proto 'static'
	option ipaddr '192.168.1.6'
	option netmask '255.255.255.0'
	option ip6assign '60'
	option gateway '192.168.1.1'
	list dns '192.168.1.1'

config device
	option name 'br-wan'
	option type 'bridge'
	list ports 'eth0.2'

config interface 'wan'
	option device 'br-wan'
	option proto 'none'

config switch
	option name 'switch0'
	option reset '1'
	option enable_vlan '1'

config switch_vlan
	option device 'switch0'
	option vlan '1'
	option ports '0t 4 5 1'
	option vid '1'

config switch_vlan
	option device 'switch0'
	option vlan '2'
	option ports '0t 2 1t'
	option vid '101'

config switch_vlan
	option device 'switch0'
	option vlan '3'
	option ports '0t 3 1t'
	option vid '102'

config device
	option name 'br-iot'
	option type 'bridge'
	list ports 'eth0.101'

config device
	option name 'br-camera'
	option type 'bridge'
	list ports 'eth0.102'

config interface 'iot'
	option device 'br-iot'
	option proto 'none'

config interface 'camera'
	option device 'br-camera'
	option proto 'none'

So, I assumed that by removing port 1 from the vlan switch setting and defining the br-wan as static IP, I should be able to ping 192.168.1.236 when connecting to the blue wan port. But there's no way to connect to that address. Configuration below

config device
	option name 'br-lan'
	option type 'bridge'
	list ports 'eth0.1'

config interface 'lan'
	option device 'br-lan'
	option proto 'static'
	option ipaddr '192.168.1.6'
	option netmask '255.255.255.0'
	option ip6assign '60'
	option gateway '192.168.1.1'
	list dns '192.168.1.1'

config device
	option name 'br-wan'
	option type 'bridge'
	list ports 'eth0.2'

config interface 'wan'
	option device 'br-wan'
	option proto 'static'
	option ipaddr '192.168.1.236'
	option netmask '255.255.255.0'
	option ip6assign '60'
	option gateway '192.168.1.1'
	list dns '192.168.1.1'

config switch
	option name 'switch0'
	option reset '1'
	option enable_vlan '1'

config switch_vlan
	option device 'switch0'
	option vlan '1'
	option ports '0t 4 5'
	option vid '1'

config switch_vlan
	option device 'switch0'
	option vlan '2'
	option ports '0t 2'
	option vid '101'

config switch_vlan
	option device 'switch0'
	option vlan '3'
	option ports '0t 3'
	option vid '102'

config device
	option name 'br-iot'
	option type 'bridge'
	list ports 'eth0.101'

config device
	option name 'br-camera'
	option type 'bridge'
	list ports 'eth0.102'

config interface 'iot'
	option device 'br-iot'
	option proto 'none'

config interface 'camera'
	option device 'br-camera'
	option proto 'none'

config switch_vlan
	option device 'switch0'
	option vlan '4'
	option ports '0t 1'

I even tried with a different port definition for wan (changed only the lines below), but nothing changes

config device
	option name 'eth0.2'
	option macaddr '74:da:88:a9:6d:c1'

config interface 'wan'
	option device 'eth0.2'
	option proto 'static'
	option ipaddr '192.168.1.236'
	option netmask '255.255.255.0'
	option gateway '192.168.1.1'
	list dns '192.168.1.1'

I'm sure I'm missing something glaringly obvious, but I can't figure out what

greybeard · December 19, 2024, 12:23pm

your second link needs to be a different network, not the same network as the primary link.
Isolate the secondary link, ie different ethernet port and network. You can change and amalgamate later.
You shouldn't need gateways on the trunk link in a simple network. You're just trying to get a trunk link working then adding a tunnel, followed by vlan traffic.
You don't want your networks to go via specific paths/links/tunnels, don't give them an alternative path.
Look at something like this example from the wiki
https://openwrt.org/docs/guide-user/network/routing/examples/routing_in_gre

robca · December 19, 2024, 7:23pm

First of all, thanks for your patience and not giving up on me.

I'm not sure I understand your last reply, sorry. In the Archer A7, as far as I understand, eth0.1 is the 4-port lan block, eth0.2 is the blue WAN port. The default network setting for the Archer A7 (as a router) is as below

config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fdaa:a9d2:3aa0::/48'

config device
	option name 'br-lan'
	option type 'bridge'
	list ports 'eth0.1'

config interface 'lan'
	option device 'br-lan'
	option proto 'static'
	option ipaddr '192.168.1.1'
	option netmask '255.255.255.0'
	option ip6assign '60'

config device
	option name 'eth0.2'
	option macaddr '74:da:88:a9:6d:c1'

config interface 'wan'
	option device 'eth0.2'
	option proto 'dhcp'

config interface 'wan6'
	option device 'eth0.2'
	option proto 'dhcpv6'

config switch
	option name 'switch0'
	option reset '1'
	option enable_vlan '1'

config switch_vlan
	option device 'switch0'
	option vlan '1'
	option ports '2 3 4 5 0t'

config switch_vlan
	option device 'switch0'
	option vlan '2'
	option ports '1 0t'

Plus the firewall definitions

config zone
	option name		lan
	list   network		'lan'
	option input		ACCEPT
	option output		ACCEPT
	option forward		ACCEPT

config zone
	option name		wan
	list   network		'wan'
	list   network		'wan6'
	option input		REJECT
	option output		ACCEPT
	option forward		REJECT
	option masq		1
	option mtu_fix		1

config forwarding
	option src		lan
	option dest		wan

So, I thought that if I want to set a new link on WAN (let's use trunk as a name), I need something like (only the relevant parts copied here)

config device
	option name 'eth0.2'
	option macaddr '74:da:88:a9:6d:c1'

config interface 'trunk'
	option device 'eth0.2'
	option proto 'static'
	option ipaddr '192.168.30.236'
	option netmask '255.255.255.0'
	option gateway '192.168.30.1'

config switch_vlan
	option device 'switch0'
	option vlan '4'
	option ports '1 0t'

My mental model is that I configure eth0.2 as a device, create an interface on that port with a static IP address (I tried a different address this time, per your suggestion), then connect that port to the switch/processor interface as untagged. I connect a PC with a static IP address in the 192.168.30.x network and ping the Archer address, with no reply. I also tried the definitions below (bridge with eth0.2, define trunk with that bridge), no difference

config device
	option name 'br-trunk'
	option type 'bridge'
	list ports 'eth0.2'

config interface 'trunk'
	option device 'br-trunk'
	option proto 'static'
	option ipaddr '192.168.30.236'
	option netmask '255.255.255.0'
	option ip6assign '60'
	option gateway '192.168.30.1'

What am I doing wrong? I know that eth0.2 works when connected to br-lan via the switch and not as a separate interface, so at least I can rule out physical problems

Your gre link makes things super clear now... as long as I can figure out a way to use etho0.2

greybeard · December 19, 2024, 10:53pm

Your firewall defaults for a WAN port have traffic blocked and masquerade enabled.
So on the default config, I'd delete any mention of wan6 (for clarity, worry about ipv6 later if required). Change the WAN interface to your test static IP/network, change the WAN firewall to allow (ie no block/denys) and disable masquerade.

The default configuration should work for your example.
Now go back and change WAN references to whatever makes sense to you, ie trunk.
If you want tagged traffic in/out of the physical interface the change the switch settings from untagged to tagged.
Repeat for additional tagged traffic on your trunk. You can mix tagged and untagged on a link but your on your own.

Look here as well https://openwrt.org/docs/guide-user/network/vlan/start

brada4 · December 19, 2024, 10:56pm

You might need conntrack helpers extra in firewall to gre across NAT

robca · December 31, 2024, 7:32pm

I could not find a way to make gretap work for my network configuration (not saying it won't work, just that I couldn't make it work )

In another forum vxlans were suggested again, and after a bit of fighting with the DSA quirks, I managed to make it all work finally.

For future readers, the trick is to use the same WAN uplink from the Archer A7 as before for the lan traffic, then create a vxlan for each vlan on my network, communicating with the main router. The rest of the devices on the network (another AP and multiple managed switches) can keep sending vlan-tagged traffic, but the Archer A7 behind the halow bridge will only send untagged traffic and use vxlans for the tagged one. Each vlan needs a separate vxlan, and the vxlan and the vlan need to be bridged on the main router side, with the vlan interface (IP address, etc) assigned to that bridge.

On the Archer, each vxlan communicates with the main router (peer address 192.168.1.1, ip address 192.168.1.6) using a unique vid. On the main router, there is the same vxlan definition just swapping ip address and peer address. The MTU of the vxlans (and the vlans bridged to it) is reduced to 1450, to allow for the 50 bytes header used by the vxlan and still fit into the 1500 MTU of the underlying lan

Here is a sample network file for the main router

config device
option name 'br-lan'
option type 'bridge'
list ports 'lan1'
list ports 'lan2'
list ports 'lan3'
list ports 'lan4'

config device
option name 'lan1'
option macaddr 'e8:9f:80:a4:15:50'

config device
option name 'lan2'
option macaddr 'e8:9f:80:a4:15:50'

config device
option name 'lan3'
option macaddr 'e8:9f:80:a4:15:50'

config device
option name 'lan4'
option macaddr 'e8:9f:80:a4:15:50'

config bridge-vlan
option device 'br-lan'
option vlan '1'
list ports 'lan1:t*'
list ports 'lan2:u*'

config interface 'lan'
option device 'br-lan.1'
option proto 'static'
option ipaddr '192.168.1.1'
option netmask '255.255.255.0'
option ip6assign '60'

config device
option name 'wan'
option macaddr 'e8:9f:80:a4:15:4f'

config interface 'wan'
option device 'wan'
option proto 'dhcp'

config interface 'wan6'
option device 'wan'
option proto 'dhcpv6'

config interface 'camera'
option device 'br-camera'
option proto 'static'
option ipaddr '192.168.20.1'
option netmask '255.255.255.0'

config interface 'vxiot'
option proto 'vxlan'
option force_link '1'
option peeraddr '192.168.1.6'
option ipaddr '192.168.1.1'
option vid '110'
option tunlink 'lan'
option mtu '1450'

config device
option type 'bridge'
option name 'br-iot'
option bridge_empty '1'
list ports 'br-lan.101'
list ports 'vxiot'

config interface 'iot'
option proto 'static'
option device 'br-iot'
option ipaddr '192.168.10.1'
option netmask '255.255.255.0'

config interface 'vxcamera'
option proto 'vxlan'
option force_link '1'
option peeraddr '192.168.1.6'
option ipaddr '192.168.1.1'
option mtu '1450'
option vid '120'
option tunlink 'lan'

config device
option type 'bridge'
option name 'br-camera'
list ports 'br-lan.102'
list ports 'vxcamera'

config bridge-vlan
option device 'br-lan'
option vlan '101'
list ports 'lan1:t'
list ports 'lan3:u*'

config bridge-vlan
option device 'br-lan'
option vlan '102'
list ports 'lan1:t'
list ports 'lan4:u*'

And the Archer A7 acting as AP

config device
option name 'br-lan'
option type 'bridge'
list ports 'eth0.1'
list ports 'eth0.2'

config interface 'lan'
option device 'br-lan'
option proto 'static'
option ipaddr '192.168.1.6'
option netmask '255.255.255.0'
option ip6assign '60'
option gateway '192.168.1.1'
list dns '192.168.1.1'

config device
option name 'eth0.2'
option macaddr '74:da:88:a9:6d:c1'

config interface 'wan'
option device 'eth0.2'
option proto 'none'

config switch
option name 'switch0'
option reset '1'
option enable_vlan '1'

config switch_vlan
option device 'switch0'
option vlan '1'
option ports '0t 4 5 1'
option vid '1'

config switch_vlan
option device 'switch0'
option vlan '3'
option ports '0t 2'
option vid '110'

config switch_vlan
option device 'switch0'
option vlan '4'
option ports '0t 3'
option vid '120'

config device
option name 'br-iot'
option type 'bridge'
list ports 'eth0.110'
list ports 'vxiot'
option bridge_empty '1'

config device
option name 'br-camera'
option type 'bridge'
list ports 'eth0.120'
list ports 'vxcamera'
option bridge_empty '1'

config interface 'iot'
option device 'br-iot'
option proto 'none'

config interface 'camera'
option device 'br-camera'
option proto 'none'

config interface 'vxiot'
option proto 'vxlan'
option peeraddr '192.168.1.1'
option ipaddr '192.168.1.6'
option vid '110'
option tunlink 'lan'
option force_link '1'
option mtu '1450'

config interface 'vxcamera'
option proto 'vxlan'
option peeraddr '192.168.1.1'
option ipaddr '192.168.1.6'
option mtu '1450'
option vid '120'
option tunlink 'lan'
option force_link '1'

system · January 10, 2025, 7:33pm

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.