Bridge multiple Lede Routers using VPN and one DHCP

Hello,
currently I am living in dorm like apartment with three other roommates. Everyone has his own Internet connection coming from a static gigabit WAN port in the wall. I wan't to create one big Wifi (same SSID) where the clients decide to which AP (Lede C7 Router) they want to connect. But I also want that all (w)lan clients can communicate with each other regardless to which AP/Router they are connected .

As I can't lay cables, haven't access to the wan Switch (providing the wall ports) and also a wifi mesh is bad idea as well (due to many wifis), I decided to realize a bridged VPN.
I think one DHCP Server without Subnets would be best, so I can walk through the apartment without long connection changes and all devices would remain their IP.

Therefore I set my router as the OpenVPN-Server and the other routers as OpenVPN-Clients. I am able to access all lan clients through the VPN but the problem is that I am loosing my Internet connection (or even sometimes I get Internet through the VPN-Server, depending on the manually restarted the services). I still want the devices to use the wan port for internet the current connected router is connected to.

To make things a little bit clearer I made a schematic:

Server Config:

etc/config/openvpn

config openvpn 'WGVPN'
option enabled '1'
option verb '3'
option proto 'udp'
option port '1194'
option dev 'tap0'
option mode 'server'
option tls_server '1'
option keepalive '10 120'
option ca '/etc/openvpn/ca.crt'
option cert '/etc/openvpn/my-server.crt'
option key '/etc/openvpn/my-server.key'
option dh '/etc/openvpn/dh2048.pem'
option status '/var/log/openvpn-status.log'

/etc/config/network

...
config interface 'lan'
option type 'bridge'
option proto 'static'
option ipaddr '192.168.1.1'
option netmask '255.255.255.0'
option ip6assign '60'
option _orig_ifname 'eth1 wlan0 wlan1 wlan1-1'
option _orig_bridge 'true'
option ifname 'eth1 tap0'
config interface 'vpn0'
option ifname 'tap0'
option proto 'none'
option auto '1'
...

Client Config:

/etc/config/openvpn

config openvpn 'WGvpn'
option enabled '1'
option proto 'udp'
option verb '3'
option ca '/etc/openvpn/ca.crt'
option cert '/etc/openvpn/my-client-115.crt'
option key '/etc/openvpn/my-client-115.key'
option client '1'
option remote_cert_tls 'server'
option dev_type 'tap'
option port '1194'
list remote ''
option dev 'tap0'
option log 'openvpn.log'
option tls_client '1'

/etc/config/network

...
config interface 'lan'
option type 'bridge'
option proto 'static'
option netmask '255.255.255.0'
option ip6assign '60'
option _orig_ifname 'eth1 wlan0 wlan1 wlan1-1'
option _orig_bridge 'true'
option ipaddr '192.168.1.3'
option ifname 'eth1 tap0'
option gateway '192.168.1.3'

config interface 'vpn0'
option ifname 'tap0'
option proto 'none'
option auto '1'
...

/etc/config/dhcp

config dhcp 'lan'
option interface 'lan'
option ra 'server'
option dhcpv6 'server'
option ra_management '1'
option ignore '1'

To mention every detail: I also have mwan3 installed as I need some policy based routing for a guest wifi using an external VPN to access the Internet

Thank you in advance

Same SSID seems impossible to determine which APP you will be using.

If you just want to form a "local" network with your roomies despite being connected to different internet routers, you may want to look into a "zero-config" vpn, like Hamachi (commercial) or NeoRouter (free, but requires your own infrastructure).

I want two things:

  1. Due to blind spots in the Apartment I also want my devices to use my roomies routers. And this should be based on the same SSID because you I can still see my own wifi (not same ssid) with a txrate~5mbit what is unusable. And wifi devices are so sticky concerning different SSIDs but not concerning a change to a different AP with same ssid and a different channel).

  2. I want a big local lan so I can stream music from my phone to the amp in the living room (different networks, no hamachi client available). Its not just for a quick gaming session in two months (usecase of Hamachi), its for music streaming (several clients), file sharing (nothing to big, so gorpo vids, backup etc.) and all that other stuff you do on your lan. I still want to access my nas even when I siting in another room.

But currently I am not concerned about the Wifi as I plug in everything on the lan. I am having problems using the routers WAN for Internet but at the same time I am able to connect to all LAN Clients (regardless to which router they are connected)

Same ssid it's your problem. You could use multiple ssid on each router and all your router share 4 wan via one cable.

As I said, I am not using the Wifi yet. Currently I am struggling with OpenVPN bridge and a own Gateway. Why should same SSID be a problem if LAN works?
To be more precise why I want one SSID: Client based roaming works poorly with aspect to switch automatically between multiple SSIDs. It works much better with the same SSID and multiple Channels for the AP (thats basically what you do for big buildings).
a[quote="felipee07, post:5, topic:2367"]
and all your router share 4 wan via one cable.
[/quote]
What do you mean? I don't have one cable, I have four. Four wan cables (for internet) going to an infrastructure I can't access.

If your internet traffic is not metered and all you want to achieve is the current bandwidth but will all local devices in one network, why not use mwan3 to aggregate all your uplinks with one router and use a number of routers as APs to broadcast the WiFi signal?

Each one has your own internet, right ! how do you expect which internet you will be using ?

If your plan is each person use own internet,multiple ssid sounds good.

Traffic is metered but I have a workaround. I don't want to use load balancing as the current bandwidth is already a gigabit connection. All I want is to see all lan clients like a big lan and use the internet of the AP I am currently connected to.

This is the point where I am not able to lay cables (rented apartment for the next years but the rooms with the wan ports in wall are roughly 20 m apart and there is no way to lay cable into one room to make another router the internet distributer (which would be awesome as everybody could still use their own internet, regardless of the connected AP)).

I am guessing this comes from the gateway config? This is more or less whats the topic about as I am currently not able to access the internet on the client side.

Nope, I dont mind my roomies using my Internet. They indeed should use it when they come to my room and their devices switch to my router.

Beside this I played with tun and different Subnets instead of tap. Then I am able to connect to the Internet and all lan clients (regardless of their subnet/router). So pretty good huh? BUT: The biggest point is AirPlay which is based on mDNS, what isn't possible for multiple Subnets. There should be a workaround with avahi, which works great on Subnets on the same router but fails over OpenVPN (I can see my Amp but connecting fails).

sorry, could be better but it still working

Right, but as I said I can't lay cables! Therefore I tried to make the green cable as VPN Bridge connection. (WAN is gigabit this should work pretty good). But I am not able to get all on the same subnet and still use the wan port.

Then what's the point of the VPN? Just use the same SSIDs with the same passwords and you are set up. No need to connect the routers with each other.

Added:
Just make sure the wireless channels of the routers are different and the DHCP servers are allocating different ranges of IP addresses. Or am I wrong and maybe there will be some kind of DHCP conflict?

Router 1:
uci set dhcp.lan.start="100"
uci set dhcp.lan.limit="30"

Router 2:
uci set dhcp.lan.start="130"
uci set dhcp.lan.limit="30"

Router 3:
uci set dhcp.lan.start="160"
uci set dhcp.lan.limit="30"

...

Thats exactly what I wanted to do at first. But then I would encouter the problem that I cant see my local lan clients which are still connect on other routers. One of the main points is airplay streaming. I want to stream music all over the house regardless of the connected AP. As Airplay works with mDNS I am forced to use a bridged VPN. (Avahi mdns reflector wont work with vpn tunnels, I tested this).
As I said the only problem is connetcing to the internetwith tap as it seems like I loose the second Routers gateway once the VPN connection is etablished.

Is using multiple DHCP server in different ranges a good idea? How would I block them to not send dhcp request through the tunnel?

I think you just need to think those 4 as remote locations between them, at first...

each router must connecto to a reliable site (can be AWS or cheaper one like DigitalOcean VPS With OpenVPN) and configured to allow to connect between enpoints (with proper public IP NAT and port redirection) I assume that probably that "WAN IP" are really ISP pseudo public IP, already NAT'ted, if you do this will automatically connect inside the ISP network without going outside...

Later on, you could leave enabled DHCP with different range (bridged OpenVPN) and diferent subnets (routed with proper routing table) depending on your needs, and you could add two wireless interfaces to each Router... one to identify it "AP 1", "AP 2", etc, and another with same name to allow Roaming and not loose connection when moving from one point to another with same name (SSID), keytype (WPA/WPA2) and key (password passphrase).

It is difficult to configure it, but in theory it is possible to do it and will work as spected.

Another way is to create a Wireless link between them, but will be possible only if you had enough signal strength beetween them, here is how I do: https://gist.github.com/braian87b/8a524a8ad74a36407a8f481e9d16a5c9

Hey there.

At a first glance, I'd say your initial setup is the way to go

  • Use TAP to make all layer2 stuff go through the line on all APs
  • Bridge the TAP port to your LAN interface on all APs, in addition to the regular eth0.1 port and the Wifi port
  • Turn off DHCP server on all APs except one

That's exactly what you described in your initial posts schematics.

As to your config files: You have two different interfaces assigned to the tap0 port. I'd suggest to skip the second one.

Since your config doesn't run, my guess is the network bridging happens when the TAP port isn't only down but not present at all. This would result in the bridge just skipping that TAP port.

Have you tried to setup up your VPN and then restart networking on your router? That might resolve that prolem since the TAP port should stay available as long as the VPN process is running. And since UDP doesn't go down when interfaces go down, the VPN connection should survive a network restart.

I'd suggest to switch from OpenVPN to TincVPN as that doesn't require a central server but can do mesh VPN. Stick with UDP and TAP devices. I cannot compare its behavior to OpenVPN, but TincVPN brings the local interface up immediately without requiring an opposite side to connect. This might bring you the marginal timing benefit of making the TAP port available for bridging in some early network states.

On top of that, just in case the "network restart" thing works, you could create a "post network script" for your VPN service. Both, OpenVPN and TincVPN can run arbitrary shell scripts once the tunnel is running. Puttng a "/etc/init.d/network restart" in there could do the trick.

Regards,
Stephan.

Another idea would be stacking interfaces. Haven't tried this, but it's worth a try.

/etc/config/network

config interface 'vpn0'
    option ifname 'tap0'
    option proto 'none'
    option auto '1'

config interface 'lan'
    option type 'bridge'
    option proto 'static'
    option ipaddr '192.168.1.1'
    option netmask '255.255.255.0'
    option ifname 'eth1 @vpn0'
    option auto '1'

Mind the @-character, which doesn't pin the linux interface tap0 but the LEDE interface vpn0.

Regards,
Stephan.

Hey there.

I just gave it a try on Virtualbox. Works like a charm.

VM1: lede-router

  • eth0: internal network "lede" (simulates switch ports on primary router)
  • eth1: NAT-Network (simulates uplink, gets 10.14.2.4 from Virtualbox DHCP)
  • eth2: management (in "lan" firewall group, just that I can SSH to the box)
  • tap0: VPN server

VM2: lede-ap:

  • eth0: internal network "vpnbridge" (simulates switch ports on foreign APs)
  • eth1: NAT-Network (simulates uplink, gets 10.14.2.5 from Virtualbo DHCP)
  • eth2: management
  • tap0: VPN client

VM3: Ubuntu

  • eth0: internal network "vpnbridge" (simulates a client device on one of the switch ports of a foreign AP or a remote Wifi client)

So Ubuntu trying to reach the internet creates that kind of flow:

  1. source local interface on Ubuntu
  2. is connected to eth0 on VM2
  3. which is bridged to tap0 on VM2
  4. which is tunneled to tap0 on VM1
  5. which is bridged to eth0 on VM1
  6. which is routed from VM1 to eth2
  7. which is the WAN uplink

DHCP and DNS always comes from VM1.

As I said, works like a charm.
Rebooting both, VM1 and VM2 takes a couple of seconds and obviously the link goes down during that time.
When I reboot VM2, which is VPN client, traffic from Ubuntu works right after VM2 is up again since VPN reconnect is done immediately.
WhenI reboot VM1, which is the VPN server, traffic from Ubuntu stalls a couple of seconds (up to 2 minutes) since VPN is reestablished from VM2.

Tweeking the VPN setting might help here, but considering how often you're going to reboot the APs that's imho not necessary.

Don't forget to open UDP port 1194 incoming on both devices.

Here's my config:

Router:

/etc/config/network
config interface 'wan'
    option proto 'dhcp'
    option ifname 'eth1'

config interface 'mgnt'
    option proto 'dhcp'
    option ifname 'eth2'

config interface 'lan'
    option type 'bridge'
    option proto 'static'
    option ipaddr '192.168.1.1'
    option netmask '255.255.255.0'
    option _orig_ifname 'eth0'
    option _orig_bridge 'true'
    option ifname '@vpn eth0'

config interface 'vpn'
    option proto 'none'
    option ifname 'tap0'
    option auto '1'
    option delegate '0'
/etc/config/firewall

In addition to existing rules, of course.

config rule
    option target 'ACCEPT'
    option src 'wan'
    option proto 'udp'
    option dest_port '1194'
    option name 'OpenVPN'
/etc/config/openvpn
config openvpn 'WGVPN'
    option enabled '1'
    option verb '3'
    option proto 'udp'
    option port '1194'
    option dev 'tap0'
    option mode 'server'
    option tls_server '1'
    option keepalive '10 120'
    option ca '/etc/openvpn/ca.crt'
    option cert '/etc/openvpn/my-server.crt'
    option key '/etc/openvpn/my-server.key'
    option dh '/etc/openvpn/dh2048.pem'
    option status '/var/log/openvpn-status.log'
    option log '/var/log/openvpn.log'

AP:

/etc/config/network
config interface 'wan'
    option proto 'dhcp'
    option ifname 'eth1'

config interface 'mgnt'
    option proto 'dhcp'
    option ifname 'eth2'

config interface 'lan'
    option type 'bridge'
    option _orig_ifname 'eth0'
    option _orig_bridge 'true'
    option proto 'dhcp'
    option ifname '@vpn eth0'
    option delegate '0'

config interface 'vpn'
    option ifname 'tap0'
    option _orig_ifname 'tap0'
    option _orig_bridge 'false'
    option proto 'none'
    option auto '1'
    option delegate '0'
/etc/config/firewall

In addition to existing rules, of course.

config rule
    option target 'ACCEPT'
    option src 'wan'
    option proto 'udp'
    option dest_port '1194'
    option name 'OpenVPN'
/etc/config/openvpn
config openvpn 'WGvpn'
    option enabled '1'
    option proto 'udp'
    option verb '3'
    option ca '/etc/openvpn/ca.crt'
    option cert '/etc/openvpn/my-client.crt'
    option key '/etc/openvpn/my-client.key'
    option client '1'
    option remote_cert_tls 'server'
    option dev_type 'tap'
    option port '1194'
    list remote '10.14.2.4'
    option dev 'tap0'
    option log '/var/log/openvpn.log'
    option tls_client '1'

You don't need the "client lan interface" which is bridged to the @vpn port to have an IP at all, nor does the @vpn interface itsef. So I made the @vpn interface unmanaged to stay neutral but then lan interface on VM2 dhcp to have it receive an IP from VM1s DHCP as soon as the VPN link is up.

As a suggestion for further improvement:

  • don't share "lan" but "roommate1" on AP1, "roommate2" on AP2 and "roommate3" on AP3 as three distinct networks.
  • Make each AP VPN server, AP1 providing its local "roommate1", AP2 providing its local "roommate2" and so on.
  • Make each AP two different VPN clients to connect to the other APs.
  • Create interfaces "roommate1" through "roommate3" on all APs, each bridging the corresponding @vpn$i interface.
  • Each of you gets his own SSID, configured on all of your APs

This allows you to move around, roam fron one AP to another but stay in your very individual network.

Regards,
Stephan.

2 Likes

Hey,
sorry for the late response. Thank you so much for your detailed answer.

By now have the following setup:
2 Wifis per Router: one with a unique SSID for Debugging or stationary items (e.g my Echo always connected to a router more far away....) and the other one with the same SSID for all of them.

The Routers are bridged using openVPN (din't saw your post at that time). At first I tried with disabled DHCP on three of them. Which worked quite well, but unfortunately the DHCP pushes the standard gateway to the clients. This resulted in the problem that all clients (regardless of the ap they are connected) used my internet/WAN interface. As this reduced speed and my WAN Traffic is limited, I made 4 individual DHCP server running on the same subnet with a dedicated range .4-.20, .21-.40. This works quite well. The biggest Problem was that the devices weren't reneawing their lease when roaming from one ap to another, therefore I set the leastime to 10m. So when I use my smartphone around the house, it uses my WAN connection most of the time which is okay. But Laptops which make the most traffic just need 10 min after going to another room. I can live with that. So all in all it is not a very beautiful solution, but one that works (more or less).

Things are getting more complicated as I have another VPN Client running on those Routers to avoid traffic Limitation on the WAN Interface.

Maybe I will try your solution but I will get the same problem, right?

Hey there.

Meanwhile I answered the very same question in the OpenWRT forum and referred to your thread here.
The discussion went on and I provided a full example.

See this thread:
https://forum.openwrt.org/viewtopic.php?id=71080

That's my setting:

It prevents that exact problem.
You basically need to pass two vlans on layer2 (bridged) throug your VPN. As soon as you manage to do that, DHCP renewal works again.

That example uses eth2/crosslink to connect the APs and provides no security at all. Use VPN if you need security, use wifi client if you have no wires (that's what my comment in that file is telling).

Regards,
Stephan.

1 Like

This looks promising, thanks. As I am currently writing my thesis I will postpone testing for a couple of months (The current setup is like a leaking boat, a couple of holes in the hull but still floating).

But just from looking at you git I am note quite sure how the clients are forced to reneaw the lease? I see that you have set lan10 and lan20 alternating static and dhcp. So in case I am connected to AP1 I am using the lan10 and then when I switch to another AP I am getting forced to use lan20? Would this mean I had to use four vlans as I have four routers.

Furthermore, after connecting to the other AP I would be on a different subnet or am I wrong? I am trying to avoid different Subnets as I want to use Airplay and stuff which is unfortunately mDNS. And the downside of mDNS is that is restricted to one subnet, and the reflectors/repeater like the avahi one doesn't work that quite well.

1 Like