Wireguard slow speed - Banana BPI-R3-Mini and OpenWrt Snapshot

hi guys,

i'm using OpenWrt 23.05 Snapshot (Kernel 6.1.82) and a wireguard tunnel from this unit (that is using a 5G connection and can reach without WG tunnel about 580/120Mbit) to my home with XGSPON 10/2 link.

If I'm enable the WG tunnel and doing an iperf both TX and RX, the speed cannot go over 10Mbps. With a client connected to the BPI-R3-Mini (iPhone), and WG tunnel back again to my home (this time with 0.0.0.0/0 to pass all traffic) I can easly reach full band of 5G)

Seems something related to this snapshot..

here is the WG configuration:

config interface 'wg0'
	option proto 'wireguard'
	option private_key 'XXXXX'
	option delegate '0'
	list addresses '172.19.255.2'
	option defaultroute '0'

config wireguard_wg0
	option description 'Home'
	option public_key 'XXXX
	list allowed_ips '0.0.0.0/0'
	option endpoint_host 'XXXX'
	option endpoint_port '64001'
	option persistent_keepalive '15'
	option route_allowed_ips '1'

firewall config:

config defaults
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option synflood_protect '1'
	option flow_offloading '1'
	option flow_offloading_hw '1'

config zone
	option name 'lan'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'
	list network 'lan'

config zone
	option name 'wan'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'
	list network 'wan'
	list network 'wan6'
	list network 'wg_wan'

config forwarding
	option src 'lan'
	option dest 'wan'

config zone
	option name 'vpn'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'
	option mtu_fix '1'
	list network 'wg0'

config forwarding
	option src 'vpn'
	option dest 'lan'

config forwarding
	option src 'vpn'
	option dest 'wan'

config forwarding
	option src 'lan'
	option dest 'vpn'

Any suggestion?

Thanks in advance

It is not entirely clear what your setup is.

Is the setup as a WireGuard "server" so that you can connect from outside to your home e.g. with your phone on cellular?

Edit:
Please connect to your OpenWRT device using ssh and copy the output of the following commands and post it here using the "Preformatted text </> " button:

Remember to redact passwords, MAC addresses and any public IP addresses you may have:

ubus call system board
cat /etc/config/network
cat /etc/config/firewall
ip route show
ip route show table all
ip rule show
wg show

BPI-R3-Mini is client, the server is Mikrotik at my home:

root@BPI-R3-Mini:~# ip route show
default via 2.192.X.X dev wwan0 proto static
2.192.X.X/30 dev wwan0 proto kernel scope link src 2.192.196.162
10.0.1.0/24 dev wg0 proto static scope link
79.53.X.X via 2.192.X.X dev wwan0 proto static
192.168.178.0/24 dev br-lan proto kernel scope link src 192.168.178.1

root@BPI-R3-Mini:~# ip route show table all
default via 2.192.196.162 dev wwan0 table pbr_wan
192.168.178.0/24 dev br-lan table pbr_wan proto kernel scope link src 192.168.178.1
default via 172.19.255.2 dev wg0 table pbr_wg0
192.168.178.0/24 dev br-lan table pbr_wg0 proto kernel scope link src 192.168.178.1
default via 2.192.X.X dev wwan0 proto static
2.192.X.X/30 dev wwan0 proto kernel scope link src 2.192.196.162
10.0.1.0/24 dev wg0 proto static scope link
79.53.x.x via 2.192.X.X dev wwan0 proto static
192.168.178.0/24 dev br-lan proto kernel scope link src 192.168.178.1
local 2.192.X.X dev wwan0 table local proto kernel scope host src 2.192.196.162
broadcast 2.192.X.X dev wwan0 table local proto kernel scope link src 2.192.X.X
local 127.0.0.0/8 dev lo table local proto kernel scope host src 127.0.0.1
local 127.0.0.1 dev lo table local proto kernel scope host src 127.0.0.1
broadcast 127.255.255.255 dev lo table local proto kernel scope link src 127.0.0.1
local 172.19.255.2 dev wg0 table local proto kernel scope host src 172.19.255.2
local 192.168.178.1 dev br-lan table local proto kernel scope host src 192.168.178.1
broadcast 192.168.178.255 dev br-lan table local proto kernel scope link src 192.168.178.1

root@BPI-R3-Mini:~# ip rule show
0:	from all lookup local
30000:	from all fwmark 0x10000/0xff0000 lookup pbr_wan
30001:	from all fwmark 0x20000/0xff0000 lookup pbr_wg0
32766:	from all lookup main
32767:	from all lookup default

root@BPI-R3-Mini:~# wg show
interface: wg0
  public key: XXXXX
  private key: (hidden)
  listening port: 33378

peer: XXXXX
  endpoint: XXXX:64001
  allowed ips: 0.0.0.0/0
  latest handshake: 1 minute, 22 seconds ago
  transfer: 710.27 KiB received, 15.49 MiB sent
  persistent keepalive: every 15 seconds
root@BPI-R3-Mini:~#

If i'm connected to this same remote server with my iPhone I can reach full 5G speed, passing both Banana BPI or directly to 5G network. The problem is when the wg is managed by the BPI-R3, there is a limit of 10-12Mbps for each type of protocol (iperf3, speedtest-cli, ftp and so on)

for instance this is an iperf3 (both normal and reverse) that pass on wg0:

[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.03  sec  3.49 MBytes  2.92 Mbits/sec  173             sender
[  5]   0.00-10.00  sec  3.25 MBytes  2.73 Mbits/sec                  receiver
[  7]   0.00-10.03  sec  3.07 MBytes  2.57 Mbits/sec  127             sender
[  7]   0.00-10.00  sec  3.00 MBytes  2.52 Mbits/sec                  receiver
[  9]   0.00-10.03  sec  3.13 MBytes  2.62 Mbits/sec  125             sender
[  9]   0.00-10.00  sec  3.00 MBytes  2.52 Mbits/sec                  receiver
[ 11]   0.00-10.03  sec  3.05 MBytes  2.55 Mbits/sec  114             sender
[ 11]   0.00-10.00  sec  3.00 MBytes  2.52 Mbits/sec                  receiver
[SUM]   0.00-10.03  sec  12.7 MBytes  10.7 Mbits/sec  539             sender
[SUM]   0.00-10.00  sec  12.2 MBytes  10.3 Mbits/sec                  receiver

and -R:

[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  3.38 MBytes  2.83 Mbits/sec  155             sender
[  5]   0.00-10.04  sec  3.28 MBytes  2.74 Mbits/sec                  receiver
[  7]   0.00-10.00  sec  4.25 MBytes  3.56 Mbits/sec  314             sender
[  7]   0.00-10.04  sec  3.87 MBytes  3.23 Mbits/sec                  receiver
[  9]   0.00-10.00  sec  3.88 MBytes  3.25 Mbits/sec  285             sender
[  9]   0.00-10.04  sec  3.68 MBytes  3.08 Mbits/sec                  receiver
[ 11]   0.00-10.00  sec  2.88 MBytes  2.41 Mbits/sec  187             sender
[ 11]   0.00-10.04  sec  2.70 MBytes  2.26 Mbits/sec                  receiver
[SUM]   0.00-10.00  sec  14.4 MBytes  12.1 Mbits/sec  941             sender
[SUM]   0.00-10.04  sec  13.5 MBytes  11.3 Mbits/sec                  receiver

During the test, checking htop, there is no CPU overhead:

And there are two speedtest, one from the BPI-R3 (client), and one from my XGSPON line:

root@BPI-R3-Mini:~# speedtest -s 3243

   Speedtest by Ookla

      Server: TIM SpA - Rome (id: 3243)
         ISP: Telecom Italia Mobile
Idle Latency:    20.55 ms   (jitter: 2.21ms, low: 18.33ms, high: 23.56ms)
    Download:   494.73 Mbps (data used: 900.0 MB)
                190.80 ms   (jitter: 59.73ms, low: 27.20ms, high: 401.18ms)
      Upload:   117.27 Mbps (data used: 117.7 MB)
                259.08 ms   (jitter: 64.78ms, low: 25.28ms, high: 485.46ms)
 Packet Loss:     0.0%
loki:~$ speedtest -s 3243

   Speedtest by Ookla

      Server: TIM SpA - Rome (id: 3243)
         ISP: Telecom Italia
Idle Latency:     5.93 ms   (jitter: 0.18ms, low: 5.79ms, high: 6.08ms)
    Download:  7659.26 Mbps (data used: 7.6 GB)
                 14.24 ms   (jitter: 18.35ms, low: 5.16ms, high: 300.12ms)
      Upload:  2057.23 Mbps (data used: 1.0 GB)
                  7.66 ms   (jitter: 0.50ms, low: 3.88ms, high: 8.71ms)
 Packet Loss:     0.0%

i'm adding also system board info:

root@BPI-R3-Mini:~# ubus call system board
{
	"kernel": "6.1.82",
	"hostname": "BPI-R3-Mini",
	"system": "ARMv8 Processor rev 4",
	"model": "Bananapi BPi-R3 Mini",
	"board_name": "bananapi,bpi-r3-mini",
	"rootfs_type": "squashfs",
	"release": {
		"distribution": "OpenWrt",
		"version": "SNAPSHOT",
		"revision": "r25869-12b2cb2ec3f1",
		"target": "mediatek/filogic",
		"description": "OpenWrt SNAPSHOT r25869-12b2cb2ec3f1"
	}
}

OK so you are using it as a travel router to connect to your home, to the mikrotik WG server?

I don not know about the wireless capacity of the banana Pi maybe that is the limiting factor but first lets review your WG setup.

You have option default route '0' meaning you are not using the tunnel at all by default which isprobabaly fine as you are using PBR.
I suggest to change your list address to: list addresses '172.19.255.2/24' (but that is not your main problem but makes sure you always have a proper route to the server side)

Your firewall is setup as if this a server, it is possible you want to use it for a site-to-site setup for that to work make sure the mikrotik has a route to this router (192.168.178.0/24?, set it in the Allowed IPs of the banana Pi peer on the Mikrotik and make sure you route allowed IPs, or make a static route).

I do not know about the wireless capabilities of the Banana Pi, so cannot comment on the maximum speed you can achieve.
But something left to play with is the MTU of the WG interface, if the MTU is too high it can have adverse effect on speed.
Default MTU is 1420 which can be too high especially if used via public Wifi.
So you can lower MTU e.g. to 1280 set this on both server and client side and see if that is better, sometimes even lower is necessary.
You can set it in the GUI WG interface > Advanced or add: option mtu '1280' to the WG interfaces of client and server.

That is basically all I can think of at this moment

It’s not portable but used as FWA in another house, and just a single client (using PBR) should use WG tunnel.

I don’t think it’s WiFi problem, because without tunnel iPhone connected to WiFi can reach full speed (with or without embedded WireGuard tunnel on iOS). Same result can be obtained using Ethernet..

I’ve also tried to lower the MTU to 1280, but still same 10-12mbits cap.

Now I’ve rebuilt the snapshot with 6.6 kernel and I’ll try to test later tonight or tomorrow..

hi all.. i've flashed the build with testing kernel (6.6) and speed now are good - same configuration with MTU 1420

[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.06  sec  56.0 MBytes  46.7 Mbits/sec   11             sender
[  5]   0.00-10.00  sec  55.4 MBytes  46.4 Mbits/sec                  receiver
[  7]   0.00-10.06  sec  19.6 MBytes  16.4 Mbits/sec    7             sender
[  7]   0.00-10.00  sec  19.2 MBytes  16.1 Mbits/sec                  receiver
[  9]   0.00-10.06  sec  24.1 MBytes  20.1 Mbits/sec   21             sender
[  9]   0.00-10.00  sec  23.6 MBytes  19.8 Mbits/sec                  receiver
[ 11]   0.00-10.06  sec   137 MBytes   114 Mbits/sec  724             sender
[ 11]   0.00-10.00  sec   135 MBytes   113 Mbits/sec                  receiver
[SUM]   0.00-10.06  sec   237 MBytes   197 Mbits/sec  763             sender
[SUM]   0.00-10.00  sec   234 MBytes   196 Mbits/sec                  receiver

So there should be an issue on 6.1 for sure..