[18.06.4] speed fix for BT HomeHub 5a

I used the 'long' 20 line 'untested fix' echo command script.

Sorry for not being clear. '(echo) & flow' was my abbreviation for using both above script and enabling software flow offloading.

2 Likes

Can you show your config, i wonder what's the cpu affinity on this 2 cpu router:
cat /proc/interrupts; grep . /proc/irq/*/smp_affinity_list

on ZBT WG3526 with MT7621A which should be 2 cores but shows 4 in /proc/cpuinfo defaults are that ethernet irqs are handled by all cpus (0-3), but each radio is pinned to a specific cpu:

# cat /proc/interrupts|grep -E "mt7|ethernet"
 22:   11376422          0          0          0  MIPS GIC  10  1e100000.ethernet
 24:         23          0          0          0  MIPS GIC  11  mt7603e
 25:          2          0   98994760          0  MIPS GIC  31  mt76x2e

# grep . /proc/irq/{22,24,25}/smp_affinity_list
/proc/irq/22/smp_affinity_list:0-3
/proc/irq/24/smp_affinity_list:3
/proc/irq/25/smp_affinity_list:2

# grep . /sys/class/net/*/queues/*/rps_cpus
/sys/class/net/br-lan/queues/rx-0/rps_cpus:0
/sys/class/net/eth0.1/queues/rx-0/rps_cpus:0
/sys/class/net/eth0.2/queues/rx-0/rps_cpus:0
/sys/class/net/eth0/queues/rx-0/rps_cpus:e
/sys/class/net/lo/queues/rx-0/rps_cpus:0
/sys/class/net/wlan0/queues/rx-0/rps_cpus:e
/sys/class/net/wlan1/queues/rx-0/rps_cpus:e

It's come to my attention that @Reiver commit has finally made it into Master and will be included in OpenWrt 20.x
https://git.openwrt.org/?p=openwrt/openwrt.git;a=commit;h=d3868f15f876507db54afacdef22a7059011a54e

Apparently, it will not be back ported into 19.07, or 18.06.
https://github.com/openwrt/openwrt/pull/2553#issuecomment-594771114

@Reiver

Judging from this:

and this:

I can see that if somebody wants to manually edit the existing "20-smp-tune" file to take advantage of the performance fix needs to copy/paste the following instead of the code suggested back in 12 Nov '19:

for q in ${dev}/queues/rx-*; do
            set_hex_val "$q/rps_cpus" "$PROC_MASK"
    done

    for q in ${dev}/queues/tx-*; do
            set_hex_val "$q/xps_cpus" "$PROC_MASK"
    done

Could you please confirm if that's the case or if the 4 additional lines that I removed are still needed? I am talking about those:

ntxq="$(ls -d ${dev}/queues/tx-* | wc -l)"
idx=$(($irq_cpu + 1))
            
let "idx = idx + 1"
[ "$idx" -ge "$NPROCS" ] && idx=0

The quickest way on an existing device, since packet steering is now disabled by default, is just to;

$ rm /etc/hotplug.d/net/20-smp-tune

Of course, this won't affect currently up interfaces until a restart or you manually echo 0 to their rps_cpus and xps_cpus proc entries.

Those four additional lines are not needed if you want to keep packet steering enabled and use the $PROC_MASK change from before (so not just non-IRQ handling cores). They're just noise now.

I've just tested latest snapshot r12470 on a spare HH5A.

Using iperf3 multithreaded testing, the best WAN to LAN I can get is 80 Mbps. This is no better than 18.06 with 'broken' packet steering settings, and far worse than 17.01.

I re-read the description of the Commit and it states packet steering is now 'disabled by default' and

The previous netifd implementation (by default but could be configured) simply used all CPUs and this patch essentially reverts to this behaviour.

But the new commit clearly doesn't return 130+ Mbps throughput for the HH5a from my observations when compared to SMP snapshots from 2017 for the HH5a.

The new commit seems to have fixed one problem but created another ?

Update: Following tip from @mkresin

To enable packet steering, SSH into the HH5A and execute these two commands:

uci set network.globals.packet_steering=1
uci commit network

This will also add a line to the /etc/config/network file:

config globals 'globals'
	option packet_steering '1'

WAN to LAN throughput now seems to return 130 Mbps in both directions in brief testing.

2 Likes

Just installed r12629 and enabled packet_steering. However I don't see any changes, my WAN to LAN speeds are still the same ~50Mbps both directions. Any ideas to debug?

I just tried r12629. I get 100-130 Mbps WAN to LAN using iperf3 multi-threaded speeds with packet steering enabled in both directions. Only 70-80 Mbps with packet steering disabled.

Single threaded iperf3 speed tests return 90 Mbps in both directions between WAN and LAN with packet steering enabled.

50 Mbps seems awfully low. How are you measuring the speed, and is it without installing any extra packages (eg. SQM) other than LuCI ?

Stating the obvious, but did you reboot router after enabling packet steering?

Hi all,
I have a BT HomeHub 5a and I am on 19.07.2. If I understood correctly, to fix this I would need to upgrade to the current snapshot version and then enable packet steering? Thanks

Not necessarily. You may simply edit /etc/hotplug.d/net/20-smp-tune as explained in the Reiver's post above.

I am on master and I have this setting in LuCI. Does this option conflict with SQM?

Nope, SQM and packet steering are entirely separate.

There is now an option in LUCI master with commit to enable or disable packet steering under Network - Interfaces - Global Network Options. Text indicates that packet steering "May help or hinder network speed" as not one size fits all.

@bill888 Have you setup proper interrupt balancing? You need the patches from topic Xrx200 IRQ balancing between VPEs, two for kernel 4.19 or one for kernel 5.4 experimental.
EDIT: Oops misread the update that your testing returned to 130 Mbps.

1 Like

I've finally flashed OpenWrt v19.07.2 onto my BT HH5 router and everything works great except for the WAN to LAN speed which I did read up beforehand. The OpenWrt Wiki points to this forum for a fix. Skimming throught the comments I saw a mention of this being implemented into the OpenWrt 20.x.

I already compile my own firmware for the Linksys EA6350v3 and WRT1900ACS and I was wondering can I safely compile OpenWrt v19.07.2 with a patch that addresses this WAN to LAN throughput issue?

Does anybody know if there is a patch I can download and apply into my own compiled firmware please?

Many thanks in advance :blush:

I bought a used, pre-OpenWRT-ed BT HH5A (thread that helped me choose it). It's on its way. I will use it in pure modem (bridge) mode: pppoe sessions initiated from the router downstream, and possibly a VLAN on a different /24 network (on the same router WAN cable if doable) to get to the modem's management interface.

What is the minimum Lede/OpenWRT that's free from the packet steering problem and as efficient as possible in WAN<->LAN throughput? Or, better stated, how do I quickly check if any necessary patches have been applied?

I guess I'm better off making a new image when 20.x comes out. From what I read here, it seems reasonable defaults will be instated. Problem is, I don't know how old the release on my device is, and 20.x is not there yet.

I expect no packet steering issue with any openwrt version running in bridge mode.
So you can keep the one already flashed when you receive the device or update to latest 19.07.3

1 Like

So even with the performance fix for the upcoming openwrt 20.x builds, Would the WAN > LAN still be capped at 130Mbits max?

Okay I get is enough for most FTTC VDSL 2 connections as of the 80/20 profile by Openreach.

I installed the r15373 snapshot a few days ago. I found that packet steering was already set to 1 in /etc/config/network.
I was curious about what the SMP hotplug script was doing so I set DEBUG=yes in the script hoping to see the debug output, but couldn't find any output anywhere (I have syslog sending to an external server and I see netifd messages in that stream, but none of the debug messages showed up). I modified the hotplug script to send output to /tmp/20-smp and then found this:

set_hex_val /sys/class/net/eth0/queues/tx-0/xps_cpus = 3
sh: write error: No such file or directory
set_hex_val /sys/class/net/eth0/queues/rx-0/rps_cpus = 3
set_hex_val /sys/class/net/wlan0/queues/tx-0/xps_cpus = 3
sh: write error: No such file or directory
set_hex_val /sys/class/net/wlan0/queues/rx-0/rps_cpus = 3
set_hex_val /sys/class/net/wlan1/queues/tx-0/xps_cpus = 3
sh: write error: No such file or directory
set_hex_val /sys/class/net/wlan1/queues/rx-0/rps_cpus = 3

The files do exist:

-rw-r--r--    1 root     root          4096 Jan  1 14:12 /sys/class/net/eth0/queues/tx-0/xps_cpus
-rw-r--r--    1 root     root          4096 Jan  1 14:12 /sys/class/net/wlan0/queues/tx-0/xps_cpus
-rw-r--r--    1 root     root          4096 Jan  1 14:12 /sys/class/net/wlan1/queues/tx-0/xps_cpus

What does the error mean?

I had also set "-l 5 -d 15" in /etc/init.d/network to try to get debug output from netifd but found nothing new in syslog.
Found that I needed to do:
procd_set_param stderr 1
in /etc/init.d/network so I did that and also did:
procd_set_param stdout 1
and I now see extra logging from netifd, but still see no messages from the SMP script.

1 Like

Hello,
so at this day what is finally the patch to apply ?? this is very confusing, for people landing here from the openwrt wiki for the BTHH5A.

Do we just have to replace the content of /etc/hotplug.d/net/20-smp-tune
with :

for q in ${dev}/queues/rx-*; do
            set_hex_val "$q/rps_cpus" "$PROC_MASK"
    done

    for q in ${dev}/queues/tx-*; do
            set_hex_val "$q/xps_cpus" "$PROC_MASK"
done

???

Thanks for helping us :slight_smile:

that's correct