[18.06.4] speed fix for BT HomeHub 5a

bill888 · January 1, 2020, 10:25am

For anyone who is interested, I ran iperf3 against LEDE 17.01.6, 18.06.4 and 19.07.0-rc2, with and without software flow offloading and fixes for 20-smp-tune script (20 line untested fix). Using Windows 10 laptops. Server connected to red WAN port (static IP) or a LAN port. Client connected to LAN port or 5 GHz wifi.

Example iperf3 commands:
Single thread: iperf3 -c 192.168.111.2 -t 10 -R
Single thread: iperf3 -c 192.168.111.2 -t 10 
Multi thread: iperf3 -c 192.168.111.2 -t 10 -P 5 -R
Multi thread: iperf3 -c 192.168.111.2 -t 10 -P 5

francisuk1989 · January 3, 2020, 1:01am

Thanks for the time for making the above results @bill888 with the 19.07.0-RC2 builds, Did you make any changes in the settings such as

WMM Mode (Did you leave enable or disabled)
Did you force 40MHz under the 2,4G band
802.11w Management Frame Protection

bill888 · January 3, 2020, 1:28am

No extra packages installed. Using Ch.36 and WPA2-PSK with CCMP(AES) enabled were only wifi changes.
1 - default. WMM Mode enabled.
2 - only 5 GHz 80 MHz tested. 2.4 GHz radio disabled.
3 - tbh, no idea what and where this 802.11w Management Frame Protection setting is located.

blackie · January 8, 2020, 7:34am

Thanks a lot for the testing. I'm not sure I get what exactly you mean by "(echo) & flow"...
Flow offload enabled + echo script is the 2 lines script from your post above?

echo 3 > /sys/class/net/eth0/queues/rx-0/rps_cpus
echo 3 > /sys/class/net/eth0/queues/tx-0/xps_cpus

or the long script version below "# Untested fix to restore to pre Feb 2018 values." ?

bill888 · January 8, 2020, 7:49am

I used the 'long' 20 line 'untested fix' echo command script.

Sorry for not being clear. '(echo) & flow' was my abbreviation for using both above script and enabling software flow offloading.

paravz · January 9, 2020, 11:54pm

Can you show your config, i wonder what's the cpu affinity on this 2 cpu router:
cat /proc/interrupts; grep . /proc/irq/*/smp_affinity_list

on ZBT WG3526 with MT7621A which should be 2 cores but shows 4 in /proc/cpuinfo defaults are that ethernet irqs are handled by all cpus (0-3), but each radio is pinned to a specific cpu:

# cat /proc/interrupts|grep -E "mt7|ethernet"
 22:   11376422          0          0          0  MIPS GIC  10  1e100000.ethernet
 24:         23          0          0          0  MIPS GIC  11  mt7603e
 25:          2          0   98994760          0  MIPS GIC  31  mt76x2e

# grep . /proc/irq/{22,24,25}/smp_affinity_list
/proc/irq/22/smp_affinity_list:0-3
/proc/irq/24/smp_affinity_list:3
/proc/irq/25/smp_affinity_list:2

# grep . /sys/class/net/*/queues/*/rps_cpus
/sys/class/net/br-lan/queues/rx-0/rps_cpus:0
/sys/class/net/eth0.1/queues/rx-0/rps_cpus:0
/sys/class/net/eth0.2/queues/rx-0/rps_cpus:0
/sys/class/net/eth0/queues/rx-0/rps_cpus:e
/sys/class/net/lo/queues/rx-0/rps_cpus:0
/sys/class/net/wlan0/queues/rx-0/rps_cpus:e
/sys/class/net/wlan1/queues/rx-0/rps_cpus:e

bill888 · March 7, 2020, 11:57am

It's come to my attention that @Reiver commit has finally made it into Master and will be included in OpenWrt 20.x
https://git.openwrt.org/?p=openwrt/openwrt.git;a=commit;h=d3868f15f876507db54afacdef22a7059011a54e

Apparently, it will not be back ported into 19.07, or 18.06.
https://github.com/openwrt/openwrt/pull/2553#issuecomment-594771114

pstlr78 · March 7, 2020, 1:05pm

@Reiver

Judging from this:

and this:

github.com

alanswanson/openwrt/blob/0a04d809afd82d22658331f5e9cacee1a67dd1c8/package/network/config/netifd/files/etc/hotplug.d/net/20-smp-packet-steering

#!/bin/sh
[ "$ACTION" = add ] || exit

NPROCS="$(grep -c "^processor.*:" /proc/cpuinfo)"
[ "$NPROCS" -gt 1 ] || exit

PROC_MASK="$(( (1 << $NPROCS) - 1 ))"

find_irq_cpu() {
	local dev="$1"
	local match="$(grep -m 1 "$dev\$" /proc/interrupts)"
	local cpu=0

	[ -n "$match" ] && {
		set -- $match
		shift
		for cur in `seq 1 $NPROCS`; do
			[ "$1" -gt 0 ] && {
				cpu=$(($cur - 1))
				break

This file has been truncated. show original

I can see that if somebody wants to manually edit the existing "20-smp-tune" file to take advantage of the performance fix needs to copy/paste the following instead of the code suggested back in 12 Nov '19:

for q in ${dev}/queues/rx-*; do
            set_hex_val "$q/rps_cpus" "$PROC_MASK"
    done

    for q in ${dev}/queues/tx-*; do
            set_hex_val "$q/xps_cpus" "$PROC_MASK"
    done

Could you please confirm if that's the case or if the 4 additional lines that I removed are still needed? I am talking about those:

ntxq="$(ls -d ${dev}/queues/tx-* | wc -l)"
idx=$(($irq_cpu + 1))
            
let "idx = idx + 1"
[ "$idx" -ge "$NPROCS" ] && idx=0

Reiver · March 7, 2020, 3:17pm

The quickest way on an existing device, since packet steering is now disabled by default, is just to;

$ rm /etc/hotplug.d/net/20-smp-tune

Of course, this won't affect currently up interfaces until a restart or you manually echo 0 to their rps_cpus and xps_cpus proc entries.

Those four additional lines are not needed if you want to keep packet steering enabled and use the $PROC_MASK change from before (so not just non-IRQ handling cores). They're just noise now.

bill888 · March 8, 2020, 10:10am

I've just tested latest snapshot r12470 on a spare HH5A.

Using iperf3 multithreaded testing, the best WAN to LAN I can get is 80 Mbps. This is no better than 18.06 with 'broken' packet steering settings, and far worse than 17.01.

I re-read the description of the Commit and it states packet steering is now 'disabled by default' and

The previous netifd implementation (by default but could be configured) simply used all CPUs and this patch essentially reverts to this behaviour.

But the new commit clearly doesn't return 130+ Mbps throughput for the HH5a from my observations when compared to SMP snapshots from 2017 for the HH5a.

The new commit seems to have fixed one problem but created another ?

Update: Following tip from @mkresin

To enable packet steering, SSH into the HH5A and execute these two commands:

uci set network.globals.packet_steering=1
uci commit network

This will also add a line to the /etc/config/network file:

config globals 'globals'
	option packet_steering '1'

WAN to LAN throughput now seems to return 130 Mbps in both directions in brief testing.

spyking · March 17, 2020, 2:12pm

Just installed r12629 and enabled packet_steering. However I don't see any changes, my WAN to LAN speeds are still the same ~50Mbps both directions. Any ideas to debug?

bill888 · March 17, 2020, 3:56pm

I just tried r12629. I get 100-130 Mbps WAN to LAN using iperf3 multi-threaded speeds with packet steering enabled in both directions. Only 70-80 Mbps with packet steering disabled.

Single threaded iperf3 speed tests return 90 Mbps in both directions between WAN and LAN with packet steering enabled.

50 Mbps seems awfully low. How are you measuring the speed, and is it without installing any extra packages (eg. SQM) other than LuCI ?

Stating the obvious, but did you reboot router after enabling packet steering?

multiforme · March 28, 2020, 6:53pm

Hi all,
I have a BT HomeHub 5a and I am on 19.07.2. If I understood correctly, to fix this I would need to upgrade to the current snapshot version and then enable packet steering? Thanks

savostyanov · March 28, 2020, 9:19pm

Not necessarily. You may simply edit /etc/hotplug.d/net/20-smp-tune as explained in the Reiver's post above.

lucenera · March 30, 2020, 11:31am

I am on master and I have this setting in LuCI. Does this option conflict with SQM?

Reiver · March 30, 2020, 8:29pm

Nope, SQM and packet steering are entirely separate.

There is now an option in LUCI master with commit to enable or disable packet steering under Network - Interfaces - Global Network Options. Text indicates that packet steering "May help or hinder network speed" as not one size fits all.

@bill888 Have you setup proper interrupt balancing? You need the patches from topic Xrx200 IRQ balancing between VPEs, two for kernel 4.19 or one for kernel 5.4 experimental.
EDIT: Oops misread the update that your testing returned to 130 Mbps.

willowen100 · April 29, 2020, 7:38pm

I've finally flashed OpenWrt v19.07.2 onto my BT HH5 router and everything works great except for the WAN to LAN speed which I did read up beforehand. The OpenWrt Wiki points to this forum for a fix. Skimming throught the comments I saw a mention of this being implemented into the OpenWrt 20.x.

I already compile my own firmware for the Linksys EA6350v3 and WRT1900ACS and I was wondering can I safely compile OpenWrt v19.07.2 with a patch that addresses this WAN to LAN throughput issue?

Does anybody know if there is a patch I can download and apply into my own compiled firmware please?

Many thanks in advance

juanriccio · May 18, 2020, 11:24am

I bought a used, pre-OpenWRT-ed BT HH5A (thread that helped me choose it). It's on its way. I will use it in pure modem (bridge) mode: pppoe sessions initiated from the router downstream, and possibly a VLAN on a different /24 network (on the same router WAN cable if doable) to get to the modem's management interface.

What is the minimum Lede/OpenWRT that's free from the packet steering problem and as efficient as possible in WAN<->LAN throughput? Or, better stated, how do I quickly check if any necessary patches have been applied?

I guess I'm better off making a new image when 20.x comes out. From what I read here, it seems reasonable defaults will be instated. Problem is, I don't know how old the release on my device is, and 20.x is not there yet.

maurer · May 18, 2020, 1:17pm

I expect no packet steering issue with any openwrt version running in bridge mode.
So you can keep the one already flashed when you receive the device or update to latest 19.07.3

francisuk1989 · May 27, 2020, 10:12pm

So even with the performance fix for the upcoming openwrt 20.x builds, Would the WAN > LAN still be capped at 130Mbits max?

Okay I get is enough for most FTTC VDSL 2 connections as of the 80/20 profile by Openreach.