OpenWrt reduces fibre speed

[GeoffatMM]

I have been using openwrt to successfully manage load balancing between 3 supplies while waiting for fibre to be installed.

Finally this week I got it and I have about 600Mb/s when I directly connect to the fibre router (free.fr). I run OpenVPN on the openwrt router using mwan3 to manage which machines use the VPN and which do not. Again, it has been running successfully for months.

The openwrt router LAN is 10.0.0.0/24. The free.fr router is 192.168.2.0/24. The WAN interface is fixed at 192.168.2.20.

When any machine connects to the LAN direct or over wifi, the bandwidth speed drops significantly. Usually between 35% and 50% of the available 600Mb/s. It can reduce further and this evening for a short while it dropped to less than 1% giving a speed of only 1.2 Mb//s.

My VPN does not appear to make any difference and often it provides a better speed but I assume that is the timing of the test and the variability of the fibre service through the openwrt router (which seems to go up and down from one second the the next).

Is there a reason this is happening and can I overcome it to get the full fibre speed through the machines, all of which connect via the LAN and not direct to the free.fr router.

Are there any logs I should post?

Thanks,

Geoff

[slh]

You don't provide any reference about the the router you expect to handle 600 MBit/s, and mwan3, and OpenVPN, and…, …and a pony…

Routing 600 MBit/s already requires serious processing power on the router, but doing OpenVPN as well bumps you firmly into quite high-end x86_64 territory.

So you have 500Mbps-1Gbps fiber and need a router READ THIS FIRST

It's such a frequently asked question over the last few years that we really need a post we can point people to. So your cable company rolled out 500Mbps or 1Gbps download speeds, or you have 1Gbps symmetric fiber (GPON/EPON) from ATT or Centurylink or Orange or whoever. You realize your old all in one router from 2009 is not up to the task of handling this, but hey by now stuff should be cheap and available that will handle your new fiber connection right? So you want advice about a router for…

[fakemanhk]

You didn't say anything about the hardware being used, there could be possibility that your router doesn't deliver enough speed even without doing anything complicated.

2nd, to troubleshoot you should reduce external factors as much as possible, take away mwan3/OpenVPN (you could probably just backup config first) and use it only with new ISP and confirm your router has adequate capability or not.

[GeoffatMM]

Thank you both for your responses. I did read the post on speed before posting and I need to provide more detail.

House burned down, in rental where I cannot install my Enigma 2 satellite receiver. Consequently, tv is over the internet. I live in France so TNT is received but for UK broadcasts I need a VPN.

The router is a cheap NewWiFi:D2 and I recognise it is not the best spec but comes loaded with Openwrt and until now has served us well. I do not want to invest in a better router as this is all a temporary solution. I am rebuilding a barn so within 18 months I will once more have satellite.

My issue was more about the massive fluctuations in speed rather than speed per se. Frankly a consistent 20 Mb/s is all we need so the reduction from 600 to 150 is OK. Fluctuating to less than 1Mb/s is not OK so something is happening on the router as these fluctuations do occur but are much less significant on the incoming ISP router.

When fibre arrived I reconfigured the router to use a single feed managed by mwan3 rules so only certain items use the VPN. I will however disable everything and run some tests and post back here.

I have checked the VPN using a client on my iPhone to bypass the OpenVPN setup and it works fine. Sadly the TV is not smart enough to use a VPN or proxy server so I have to run Openwrt to manage it.

Finally, the underlying reason for this post is that since setting up the fibre connection iPlayer runs for about 5 minutes then disconnects. I can force a reconnection or if I leave it, it reconnects itself after about 5 more minutes. Currently a one hour programme is taking over two hours to watch which is not at all enjoyable.

Last night I watched a film via the smart TV and my Prime account (running through the Openwrt router). No problems whatsoever.

So disconnection is either the router playing up or my VPN server but watching over the VPN via my iPhone is also faultless. Consequently I believe I have narrowed it down to the Openwrt router. Or mwan3. Or my interface setup. Most likely candidate being the VPN client but why has it just started?

With all this additional information are there any further suggestions you can make?

G

[slh]

mt7621 can only reach 600 MBit/s with hardware flow-offloading, I don't know if mwan3 'breaks' the offliading though (likely, because it complicates the rules and forces in legacy iptables, while offloading only works with nftables).

OpenVPN however is very CPU intensive, far beyond the abilities of mt7621 at those speeds (and this cannot be offloaded), probably 40-60 MBit/s max (and that also affects the nirmal routing performance). If your remote ends supports it, wireguard would be much easier on the CPU - by far not enough to get anywhere close to 600 MBit/s, but it should at least double the figures relative to OpenVPN.

Using a fresh install without mwan3 and fw3/ iptables, but fw4/ nftables instead might help with the offloading.

In general, your router is far too slow for your requirements, neither of the above can really fix that - but it might slightly improve the situation nevertheless.

[GeoffatMM]

Thanks. First I will remove mwan3 and reinstall OpenVPN routing. If that does not help I will do as you suggest and do a fresh install.

Never worked with wireguard but will research it. The only issue is its comparability with OpenVPN that runs on my server. I will investigate.

We can at least get a good service on my iMac 24 which bypasses the router VPN by running a Client that connects direct.

Thanks for you help.

Geoff

[nikito7]

Hardware offload: 920Mb/s
Without offload: 600Mb/s

[GeoffatMM]

Thanks to all for your help.

I discovered that the default gateway had not been changed when I renamed the interface as I changed from ADSL to Fibre so my error. Connection is now stable as a result.

For fun I built a new VPN server and after a bit of fiddling with the configuration files it is working and appears much more stable that the previous version I used.

Still getting fluctuations in speed and it is still way down from the incoming fibre but it is generally between 40Mb/s and 150Mb/s which is more than adequate for our purposes. I will live with the speed degradation.

I have moved most items to the fibre wifi on a direct connection which was quick for the phones and computers but took ages with our sonos system! The items that use the VPN remain on the original 10.0.0 network.

I removed mwan3 and reinstalled ovpn policy-routing and it all seems to work so I am happy to close this post.

Geoff

[GeoffatMM]

Well I guess I spoke too soon!

It worked fine last night and this morning (after I disconnected power from the router and restarted it.

However, by this afternoon it has once more gone haywire. At one point I recorded 0.01 Mb/s on the 10.0.0.0/24 network.

I am confident with my VPN. When I connect direct to the ISP box and use my iPhone I can connect easily to my VPN. So the VPN does not appear to be the issue.

However the internet connectivity is suffering. Once again, after working for some time, everything stops (on all connected machines) and there is a delay of some minutes before it starts itself up again. At the same time, machines connected direct to the incoming fibre are fine and high speeds are recorded. So something is choking the Internet at the entry to the router from the fibre. but why is it only doing it from time to time? If the CPU was overloaded why does that cause the disconnection of the internet?

despite the OpenVPN working fine, I am currently investigating, understanding, installing and managing the wireguard option which appears a bit more complex so I hope I can get it to work to try. Do I need to uninstall the OpenVPN or simply disable it (I assume they would conflict with one another?).

A bit of a pain but educational.

G

[jdwl1o1]

How is the isp router configured? In bridge mode or do you have double nat?

If you remove the openwrt router and use the internet with the isp router only do you see similar issues?

If not, set up monitoring on your openwrt router and then check what the cpu and ram are doing when you have problems.

[GeoffatMM]

Thanks jdwl101. I am looking at wireguard but I am not certain that it is openVPN causing the issue. I am trying to understand why it worked perfectly this morning and reverted to breaking down this evening.

Tonight I ran my VPN on the Mac using the incoming fibre internet direct and a OpenVPN client to attach to the VPN to watch TV and it worked flawlessly so I strongly believe the issue is in my setup at the interface between the two router networks. I know that double NAT can be an issue but not sure if I have it. I can see NAT anti-leakage rules in the firewall on Openwrt so I assume that means NAT is running.

Like so many ISP routers it is impossible to see into the configuration of the machine but I would normally assume it is running NAT so it is possible there is a double NAT.

The problem appears to me to be the interface between the ISP router and the WAN interface on my Openwrt router. Tonight it is so bad I am having to move from one wifi source to the other in order to interrogate the net (ISP direct) and experiment with Luci (Openwrt router where the internet is constantly dropping).

I can (and later tonight I will ) put the router in Bridge mode. Here though, are the warnings that are given when I start to do it:

Le mode bridge est recommandé aux personnes voulant utiliser leur propre routeur derrière leur connexion Free. Si vous activez ce mode, l'adresse IP publique assignée à votre connexion sera attribuée via DHCP à la première machine connectée sur les ports situés à l'arrière de votre Freebox Server.

The Bridge mode is for people with their own router behind the ISP router and the IP address will go to the first device after the ISP box.. So that is fine.

Si vous activez le mode bridge sur votre Freebox Server, cela va altérer le fonctionnement des services suivants :

If you activate Bridge the following services will be altered.

• Le service UPnP A/V sera désactivé. UPnP A/V will be disactivated. OK
• Le service UPnP IGD sera désactivé. * Ditto UPnP IGD* Not sure what this is.
• Le réseau personnel Wi-Fi du Freebox Server sera désactivé. * Ditto personal wifi.* Not sure what this is but if it is the router wifi it will give me some problems but all of them I can cope with. I will however have to reset all my Sonos equipment AGAIN!
• Le service proxy WOL fourni par le Freebox Server sera désactivé. Ditto Proxy Wakeup on LAN. I have never really understood what this is for so it is unlikely to cause me a problem.
• Le partage Windows du Freebox Server ne sera plus accessible via le voisinage réseau (mais toujours accessible via \mafreebox.freebox.fr). * The Windows partition on the Freebox server cannot be access directly but will be available via the server's domain name.*. Windows?! Windows?! Who do they think they are dealing with?
• Les services de découverte (mDNS) des partages Mac OS seront inopérants. OK. I do not use this service.
• Le ou les boitiers Freebox Player ne seront plus accessibles sur le LAN (ceci impacte uniquement le FreeStore et le navigateur Web). It will cause me an issue with the TV server but limited to their store access and their web navigator. Not a problem.
• Le ou les boitiers Freebox Player Pop n’auront plus de TV sans connectivité IPv6 SLAAC. The pop box (a TV server linked to the router server, usually over wifi) will only work over an IPv6 SLAAC connection. * No idea if this will affect me or not. We do not use their TV supply as we already have the same service over the aerial.
• Le service SeedBox sera désactivé. * Ditto for the SeedBox service.. No idea what it is yet and I am sure I do not need it.

Êtes-vous sûr de vouloir passer en mode Bridge ? * Are you really, really sure?* Yes because it can always be reversed.

So here goes. I will post results a bit later.

G

[GeoffatMM]

Too late when I finished last night to post but here is the current situation.

I tried Bridge, lost wifi and could not discover the ip addresses from the ISP router (I should explain that I no longer have any equipment with ethernet adapters so a direct connection to the router was not possible).

I read a lot, tried a lot, changed a lot but could not find a solution that worked. Then I read a post that reminded me that previously I had used a DMZ in a similar situation so tried that (this morning). It worked straight away. Tonight it was flawless.

I would love to learn more about setting up a bridged router but a bridge knocks out the wifi on the ISP router and that is a problem that does not occur using DMZ. However I also recognise that running DMZ may be dangerous wrt hackers and robots so I would also appreciate any advice on how to strengthen the interface against attack.

Geoff

[GeoffatMM]

Hello again.

Well most of the time things work but from time to time I still encounter problems so I have been looking at the logs and found that LAN Bridge did not appear to be resetting properly wrt the OpenVPN settings:

Mon Apr 17 07:55:36 2023 user.notice vpn-policy-routing [6510]: Creating table 'lan/br-lan/10.7.205.1' [✗]
Mon Apr 17 07:55:37 2023 user.notice vpn-policy-routing [6510]: Creating table 'wan_sfr/lan4/0.0.0.0' [✓]
Mon Apr 17 07:55:39 2023 user.notice vpn-policy-routing [6510]: Creating table 'ovpn_wan_free/tun0/10.0.10.3' [✓]
Mon Apr 17 07:55:40 2023 user.notice vpn-policy-routing [6510]: Creating table 'wan_fbx/wan/192.168.2.1' [✓]
Mon Apr 17 07:55:41 2023 user.notice vpn-policy-routing [6510]: Routing 'HiSenseTV' via ovpn_wan_free [✓]
Mon Apr 17 07:55:41 2023 user.notice vpn-policy-routing [6510]: Routing 'Humax' via ovpn_wan_free [✓]
Mon Apr 17 07:55:41 2023 user.notice vpn-policy-routing [6510]: Routing 'Enigma2' via ovpn_wan_free [✓]
Mon Apr 17 07:55:41 2023 user.notice vpn-policy-routing [6510]: Routing 'FireStick' via ovpn_wan_free [✓]
Mon Apr 17 07:55:42 2023 user.notice vpn-policy-routing [6510]: Routing 'GJJiMac24' via ovpn_wan_free [✓]
Mon Apr 17 07:55:42 2023 user.notice vpn-policy-routing [6510]: service monitoring interfaces: lan wan_sfr ovpn_wan_free wan_fbx [✓]
Mon Apr 17 07:55:42 2023 user.notice vpn-policy-routing [6510]: service started with gateways: wan_sfr/lan4/0.0.0.0 ovpn_wan_free/tun0/10.0.10.3 [✓] wan_fbx/wan/192.168.2.1
Mon Apr 17 07:55:42 2023 user.notice vpn-policy-routing [6510]: ERROR: Failed to set up 'lan/br-lan/10.7.205.1'

[BTW, the wan_sfr/lan4/0.0.0.0 is currently disabled]

Now 10.7.205.102 is the external IP address allocated to the fibre box and I have assumed that 10.7.102.1 is the gateway I should choose but I get the above error. I have set the fibre box to router mode with an IP of 192.168.2.1 on the internal side and am using a DMZ set to 192.168.2.20 for the Openwrt router so tried that as the gateway but that also gives the same error. I have also tried 192.168.2.1 with the same result.

Having said all the above, the system appears to be working so perhaps this is not important but any ERROR worries me!

The error can also be seen on the OpenVPN routing page:

No matter what address I enter for the gateway, the LAN rejects it so what am I doing wrong and how do I correct it please?

Geoff