Site to Site VPN with Trunked VLANS over Wireguard/GRETAP

Installing and Using OpenWrt
1

It's been a couple years. I blink and the world has changed. New technologies and DSA has changed the landscape of configuration. It's time to try a new setup to accommodate some unique network devices (such as those from Rockwell Automation, Siemens, and the like).

Hardware:

PCEngines APU2 <----------------------Fiber Internet ---------------------> PCEngines APU2
| |
| Eth. | Eth.
| |
Ubiquiti US-48-500 Ubiquiti US-48-500

APU2 eth3 is connected to Ubiquiti US-48-500 port #2 at both ends.
APU2 eth0 is connected to the fiber modem in passthrough mode on both ends as WAN.

Problem Statement:

I am looking to establish a trunked VLAN setup between 2 sites over Wireguard using GRETAP. There will be three VLANS for LAN, Guest, and IOT. Each VLAN will be provisioned 192.168.1.0/24, 192.168.2.0/24, and 192.168.3.0/24 respectively.

Presuming I can set up the Ubiquiti managed switches appropriately for the VLANS and that I can follow Fabian Lee's setup here (https://fabianlee.org/2023/01/22/openwrt-bridge-vlan-filtering-for-openwrt-21-x-with-dsa-isolated-guest-wi-fi/) and the applicable parts of Cydergoth's post here (https://github.com/cydergoth/openwrt_vlan/tree/master) how is it that we appropriately configure Wireguard and GRETAP to trunk the VLAN traffic?

Further how do we configure the WAN at both sites so the LAN and guest traffic can reach the internet?

I have read about two approaches to the site to site link here (https://gist.github.com/zOrg1331/a2a7ffb3cfe3b3b821d45d6af00cb8f6) and here (https://geek-town.com/gretap-over-wg/) but I am not clear on an exact config.

Any thoughts, tips, or complete configurations would be welcome.

2

Devices like the APU (and almost all x86) have individual ports directly connected to the CPU, so DSA does not apply. Invoke VLANs on a port the same way it has always been done, with the notation eth0.3 for example. Software bridges would be used to "switch" ports together. Use a separate bridge for each VLAN rather than bridge-vlans that are used with DSA.

Wireguard operates at layer 3 so the concept of trunking VLANs through it is not applicable. Instead you route networks through it.

1 Like
3

OWRT Layout
OWRT Layout1143×624 27.5 KB

I am going to pattern this solution roughly after the post here and try to adapt it as best I know how. Credit for the graphic heavily edited to the OP from the linked post.(Understand GRE and connect two houses - #7 by Anaaa)

This is the setup that I want to pilot for now getting toward the original goal. This is more proof of concept than functional at this stage. I will add in VLAN at some point in the next week.

If anyone sees a problem correct me.

APU2 (Site 1) Configs:

/etc/config/network

config interface 'loopback'
        option ifname 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'fd71:e596:c363::/48'

config interface 'lan'
        option type 'bridge'
        option proto 'static'
        option ipaddr '192.168.1.1'
        option netmask '255.255.255.0'
        option ip6assign '60'
        option ifname 'eth3 @gt'

config interface 'wan'
        option ifname 'eth0'
        option proto 'dhcp'
        option broadcast '1'

config interface 'wan6'
        option ifname 'eth1'
        option proto 'dhcpv6'
        option reqprefix 'auto'
        option reqaddress 'try'
        option auto '0'

config interface 'wg0'
        option proto 'wireguard'
        option private_key 'eN52dObMC1M+7gTdQwoy0+vbopS9METSKeVZas3i71o='
        option listen_port '51820'
        list addresses '192.168.100.1/24'

config wireguard_wg0 'wgclient'
        option public_key 'Q/22JOc/Wnj97WwT0tcUVBjP+oBrMeVxxxN00Pa1Xkk='
        option preshared_key 'g9fvBSHmElexqNoqoT74ONa/zmN8bV8T5+cMxEYWj+8='
        list allowed_ips '192.168.100.2/32'
        option persistent_keepalive '25'
        option route_allowed_ips '1'
        option description 'APU2 Site 1'

config interface 'gt'
        option proto 'gretap'
        option peeraddr '192.168.100.2'
        option ipaddr '192.168.100.1'

config route 'APU2S2'
        option interface 'lan'
        option target '192.168.2.0'
        option netmask '255.255.255.0'
        option gateway '192.168.100.1'

/etc/config/firewall

config defaults
        option syn_flood '1'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'REJECT'

config zone
        option name 'lan'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'ACCEPT'
        option network 'lan'

config zone
        option name 'wan'
        option input 'REJECT'
        option output 'ACCEPT'
        option forward 'REJECT'
        option masq '1'
        option mtu_fix '1'
        option network 'wan wan6 wg0'

config forwarding
        option src 'lan'
        option dest 'wan'

config rule
        option name 'Allow-DHCP-Renew'
        option src 'wan'
        option proto 'udp'
        option dest_port '68'
        option target 'ACCEPT'
        option family 'ipv4'

config rule
        option name 'Allow-Ping'
        option src 'wan'
        option proto 'icmp'
        option icmp_type 'echo-request'
        option family 'ipv4'
        option target 'ACCEPT'

config rule
        option name 'Allow-IGMP'
        option src 'wan'
        option proto 'igmp'
        option family 'ipv4'
        option target 'ACCEPT'

config rule
        option name 'Allow-DHCPv6'
        option src 'wan'
        option proto 'udp'
        option src_ip 'fc00::/6'
        option dest_ip 'fc00::/6'
        option dest_port '546'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-MLD'
        option src 'wan'
        option proto 'icmp'
        option src_ip 'fe80::/10'
        list icmp_type '130/0'
        list icmp_type '131/0'
        list icmp_type '132/0'
        list icmp_type '143/0'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-ICMPv6-Input'
        option src 'wan'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        list icmp_type 'router-solicitation'
        list icmp_type 'neighbour-solicitation'
        list icmp_type 'router-advertisement'
        list icmp_type 'neighbour-advertisement'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-ICMPv6-Forward'
        option src 'wan'
        option dest '*'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-IPSec-ESP'
        option src 'wan'
        option dest 'lan'
        option proto 'esp'
        option target 'ACCEPT'

config rule
        option name 'Allow-ISAKMP'
        option src 'wan'
        option dest 'lan'
        option dest_port '500'
        option proto 'udp'
        option target 'ACCEPT

config rule 'wg'
        option name 'Allow-WireGuard'
        option src 'wan'
        option dest_port '51820'
        option proto 'udp'
        option target 'ACCEPT'

APU2 (Site 2) Configs:

/etc/config/network

config interface 'loopback'
        option device 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'fd0f:a381:2f53::/48'

config device
        option name 'br-lan'
        option type 'bridge'
        list ports 'gt'
        list ports 'eth3'

config interface 'lan'
        option device 'br-lan'
        option proto 'static'
        option ipaddr '192.168.2.1'
        option netmask '255.255.255.0'
        option ip6assign '60'

config interface 'wan'
        option device 'eth0'
        option proto 'dhcp'

config interface 'wg0'
        option proto 'wireguard'
        option private_key 'gENATB6ik8UJYvekkP9/AXqQp2tLAz1NBzKsWKMoVFI='
        list addresses '192.168.100.2/32'

config wireguard_wg0 'wgserver'
        option public_key 'J0QU8j4E8n6s6+nrZ5yYypUDLRwJXdX7T3aDWxANLmc='
        option preshared_key 'g9fvBSHmElexqNoqoT74ONa/zmN8bV8T5+cMxEYWj+8='
        option endpoint_host 'Redacted TLD'
        option endpoint_port '51820'
        option route_allowed_ips '1'
        option persistent_keepalive '25'
        list allowed_ips '192.168.100.1/32'

config interface 'gt'
        option proto 'gretap'
        option peeraddr '192.168.100.1'
        option ipaddr '192.168.100.2'


config route 'APU2S1'
        option interface 'lan'
        option target '192.168.1.0'
        option netmask '255.255.255.0'
        option gateway '192.168.100.2'

/etc/config/firewall


config defaults
        option syn_flood '1'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'REJECT'

config zone
        option name 'lan'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'ACCEPT'
        list network 'lan'

config zone
        option name 'wan'
        option input 'REJECT'
        option output 'ACCEPT'
        option forward 'REJECT'
        option masq '1'
        option mtu_fix '1'
        list network 'wan'
        list network 'wan6'
        list network 'wg0'

config forwarding
        option src 'lan'
        option dest 'wan'

config rule
        option name 'Allow-DHCP-Renew'
        option src 'wan'
        option proto 'udp'
        option dest_port '68'
        option target 'ACCEPT'
        option family 'ipv4'

config rule
        option name 'Allow-Ping'
        option src 'wan'
        option proto 'icmp'
        option icmp_type 'echo-request'
        option family 'ipv4'
        option target 'ACCEPT'

config rule
        option name 'Allow-IGMP'
        option src 'wan'
        option proto 'igmp'
        option family 'ipv4'
        option target 'ACCEPT'

config rule
        option name 'Allow-DHCPv6'
        option src 'wan'
        option proto 'udp'
        option src_ip 'fc00::/6'
        option dest_ip 'fc00::/6'
        option dest_port '546'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-MLD'
        option src 'wan'
        option proto 'icmp'
        option src_ip 'fe80::/10'
        list icmp_type '130/0'
        list icmp_type '131/0'
        list icmp_type '132/0'
        list icmp_type '143/0'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-ICMPv6-Input'
        option src 'wan'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        list icmp_type 'router-solicitation'
        list icmp_type 'neighbour-solicitation'
        list icmp_type 'router-advertisement'
        list icmp_type 'neighbour-advertisement'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-ICMPv6-Forward'
        option src 'wan'
        option dest '*'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-IPSec-ESP'
        option src 'wan'
        option dest 'lan'
        option proto 'esp'
        option target 'ACCEPT'

config rule
        option name 'Allow-ISAKMP'
        option src 'wan'
        option dest 'lan'
        option dest_port '500'
        option proto 'udp'
        option target 'ACCEPT'

config rule
        option name 'Support-UDP-Traceroute'
        option src 'wan'
        option dest_port '33434:33689'
        option proto 'udp'
        option family 'ipv4'
        option target 'REJECT'
        option enabled '0'

config include
        option path '/etc/firewall.user'

config rule 'wg'
        option name 'Allow-WireGuard'
        option src 'wan'
        option dest_port '51820'
        option proto 'udp'
        option target 'ACCEPT'
4

You're making this more complicated than it is.

There's no need for GRE here as the two lans are different IP subnets-- simple ordinary IP layer 3 routing directly through a Wireguard tunnel will work here. Remove everything related to GRE. Forget that you even heard of GRE for now. You only need Wireguard to deploy this use case.

In a point to point link (meaning there are only two sites, not multiple peers), the tunnel itself doesn't strictly need IP addresses but it is very useful to assign them for troubleshooting with pings, and if you want to send GRE through the tunnel later you will need an IP at each terminus. These IPs must be in the same subnet (usually a /24) with the tunnel IP of the other peer(s), but outside any other subnets that are otherwise in use in any of your networks. In this case something like 192.168.200.1 and 192.168.200.2 would be suitable.

Define the other end's tunnel IP as an allowed_ip with a /32 size. Also the other side's LAN is an allowed IP on each end. Then route_allowed_ips will install the routes for you.

The wireguard interfaces can be in the lan firewall zone if you are willing to trust everything in both LANs, which can be assumed for initial testing. Since the default for the lan zone is to ACCEPT intra-zone forwarding, there is no need to configure any other forwarding for the site-site to work.

4 Likes
5

Ultimately I am looking to bridge three to four VLANS between the two sites to get my automation equipment off my home network, isolate houseguests from everything else, keep my automation equipment off the internet by zone, and have approximately four different subnets.

You're making this more complicated than it is.

Not trying to reinvent the wheel. Just trying to overcome long term obstacles and clean up the network.

There's no need for GRE here as the two lans are different IP subnets-- simple ordinary IP layer 3 routing directly through a Wireguard tunnel will work here. Remove everything related to GRE. Forget that you even heard of GRE for now. You only need Wireguard to deploy this use case.

I have a use case with PLC equipment that needs certain L2 functionalities that Wireguard alone cannot provide. This is why I am interested in GRETAP as a solution. We are talking RSLinx Classic at a bare minimum for some very backwards and legacy automation solutions and at worst site to site communication for Siemens remote racks. Unless you have a better solution (i.e. an alternate, secure approach for linking the two sites that allows the correct L2 connectivity)...

In a point to point link (meaning there are only two sites, not multiple peers), the tunnel itself doesn't strictly need IP addresses but it is very useful to assign them for troubleshooting with pings, and if you want to send GRE through the tunnel later you will need an IP at each terminus. These IPs must be in the same subnet (usually a /24) with the tunnel IP of the other peer(s), but outside any other subnets that are otherwise in use in any of your networks. In this case something like 192.168.200.1 and 192.168.200.2 would be suitable.

Will edit the applicable configurations from 192.168.100.1 or .2 to 192.168.200.1 or .2.

Define the other end's tunnel IP as an allowed_ip with a /32 size. Also the other side's LAN is an allowed IP on each end. Then route_allowed_ips will install the routes for you.

When you say the other side's LAN we're talking 192.168.1.0/24 and 192.168.2.0/24 respectively? I want to make sure this will work.

The wireguard interfaces can be in the lan firewall zone if you are willing to trust everything in both LANs, which can be assumed for initial testing. Since the default for the lan zone is to ACCEPT intra-zone forwarding, there is no need to configure any other forwarding for the site-site to work.

I've been trying for two years to get a stable Wireguard configuration that works. I can always get my cellular phone to connect successfully as a single point to point tunnel but I've never been able to get Wireguard to either provide site to site from the LAN side nor be able to correctly tunnel the traffic between two LAN bridges for pure IP traffic. The resultant setups either cause a loop evidenced by the excessive bandwidth consumption on both ends or successfully connect both networks but do not allow egress from the LAN to the internet. Therefore I am placing the Wireguard interface as part of the WAN in an effort to directionalize the traffic.

6

For a site-to-site setup do not set 0.0.0.0/0 as allowed ips.

Just set the wgtunnel address of the other side with /32 and the subnet(s) of the other side.
Make sure to enable route allowed ips

Just add the wg interface to the lan zone and you should be good

7

Have you considered L2TPv3 Pseudowire https://openwrt.org/docs/guide-user/services/vpn/pseudowire.

Looks like providing full L2 functionality over IP transport.