[18.06.4] speed fix for BT HomeHub 5a

maurer · October 21, 2018, 6:24am

It seems commit 916e33fa1e14b97daf8c9bf07a1b36f9767db679 degraded the wan-lan speed for the BT HomeHub 5a (HH v5a)

openwrt ebilan forum member bill did the following tests

iperf3 results:

20180228 Openwrt r6331-916e33f netifd update to the latest version, rewrite RPSXPS
D 80.9 Mbps, U 93.4 Mbps

the file below is a 18.06.1 with netifd reverted to 810659a

bill's results:

iperf3 results:

OpenWrt 18.06.1 r7258-5eb055306f with LuCI, by 'maurer'
eth D 142 Mbps, U 143 Mbps
N/40MHz(300Mbps) D 70 Mbps, U 91 Mbps

OpenWrt 18.06.2  with LuCI, by 'maurer'
eth D 88-131 Mbits/sec,  U 115-145 Mbits/sec

18.06.4 - Maurer edition - fixed
Download  128 Mbits/sec, Upload 136 Mbits/sec

compared to original stable release:

iperf3 results:

OpenWrt 18.06.1 r7258-5eb055306f

(Unmodified)
eth D 79.3 Mbps, U 74.3 Mbps
N/40MHz(300Mbps) D 71.8 Mbps, U 81.4? Mbps

for more details see: https://openwrt.ebilan.co.uk/viewtopic.php?f=7&t=1105

the issue is tracked in:
https://bugs.openwrt.org/index.php?do=details&task_id=1852

my 18.06.1 build with speed fix for BT HH 5a:
sha256: 26704f045fd710e199ef03578b3f60fa02594ac90f07725af6c1bfb38c1f2d64
https://anonfile.com/lbn8G1i9be
mirror: https://bit.ly/2XXGtGI

my 18.06.2 build with speed fix for BT HH 5a:
sha256: 410f4d2c9ebd104d031886cde0c70499901c7646d1d3167fb014812671ae4a3e
https://anonfile.com/q3b9iau8b5
mirror: https://bit.ly/2XRA5Ay

my 18.06.4 build with speed fix for BT HH 5a:
sha256: 931067bac645bcc6018789cf1be66dcf09d17c0d7958121a56c9c03f9c57e687
https://anonfile.com/JeV2mey5n5
mirror: https://bit.ly/2xypJqv

The only difference from a official openwrt build is a patched netifd
source code here: https://github.com/maurerr/netifd

Thanks to post your results below

blackie · October 23, 2018, 6:21am

Do you think the problem is also affecting similar Lantiq device, like TP-Link TD-W8970 ?

maurer · October 23, 2018, 6:38am

it should affect all the multi core Lantiq devices

francisuk1989 · February 9, 2019, 10:17pm

@maurer - Do you know if this has been fixed in the 18.06.2 (r7676) what was released january 2019 If so could you provide a iperf3 speedtest compared to 18.06.1 (r7258)
Thanks

maurer · February 10, 2019, 5:59pm

as you can see in the bug above there is no mention of a fix.
i hope i'll have time next week to compile 18.06.2 with the fix

LE - build is in the 1st post

browner · April 8, 2019, 6:40am

Thanks very much for these files but I'm getting errors with your 18.06.2 image

Invalid sysupgrade file.
Image check 'platform_check_image' failed.

maurer · April 8, 2019, 6:52am

please go to https://openwrt.ebilan.co.uk/viewtopic.php?f=7&t=1105 and report this

maurer · July 7, 2019, 6:12am

added 18.06.4 build tested by bill
also mirrored the files on my dropbox

solidus1983 · August 4, 2019, 10:26pm

Your anonfile links are getting flagged btw for HTML/ScrInject.B trojan you might want to check.
However your mirror links are working fine, i take it is a simple sysupgrade run for these files?

maurer · August 5, 2019, 3:58am

Thanks for the heads up. I'll reupload the flagged files when I'll have some spare time.
Yes the files are simple sysupgrades

UPDATE:
please see virustotal analysis below - NO VIRUS DETECTED !
18.06.1
18.06.2
18.06.4

bill888 · October 29, 2019, 1:23pm

fyi, bug report raised for this old issue following new findings.
https://bugs.openwrt.org/index.php?do=details&task_id=2573

Following a suggestion by mkresin to create a new script, I thought I'd add the following 2 lines to the BOTTOM of the existing /etc/hotplug.d/net/20-smp-tune script 'seems' to work and it survives hub restarts for unmodified 18.06.4 and 19.07.0-rc1 when briefly tested.

echo 3 > /sys/class/net/eth0/queues/rx-0/rps_cpus
echo 3 > /sys/class/net/eth0/queues/tx-0/xps_cpus

I haven't looked at the wireless settings and performance

# Untested fix to restore to pre Feb 2018 values.

#  Receive Packet Steering
echo 3 >   /sys/class/net/br-lan/queues/rx-0/rps_cpus
echo 3 >   /sys/class/net/eth0.1/queues/rx-0/rps_cpus
echo 3 >   /sys/class/net/eth0.2/queues/rx-0/rps_cpus
echo 3 >   /sys/class/net/eth0/queues/rx-0/rps_cpus
echo 3 >   /sys/class/net/lo/queues/rx-0/rps_cpus
echo 0 >   /sys/class/net/wlan0/queues/rx-0/rps_cpus
echo 0 >   /sys/class/net/wlan1/queues/rx-0/rps_cpus

#  Transmit Packet Steering
echo 3 >   /sys/class/net/br-lan/queues/tx-0/xps_cpus
echo 3 >   /sys/class/net/eth0.1/queues/tx-0/xps_cpus
echo 3 >   /sys/class/net/eth0.2/queues/tx-0/xps_cpus
echo 3 >   /sys/class/net/eth0/queues/tx-0/xps_cpus
echo 3 >   /sys/class/net/lo/queues/tx-0/xps_cpus
echo 0 >   /sys/class/net/wlan0/queues/tx-0/xps_cpus
echo 0 >   /sys/class/net/wlan0/queues/tx-1/xps_cpus
echo 0 >   /sys/class/net/wlan0/queues/tx-2/xps_cpus
echo 0 >   /sys/class/net/wlan0/queues/tx-3/xps_cpus
echo 0 >   /sys/class/net/wlan1/queues/tx-0/xps_cpus
echo 0 >   /sys/class/net/wlan1/queues/tx-1/xps_cpus
echo 0 >   /sys/class/net/wlan1/queues/tx-2/xps_cpus
echo 0 >   /sys/class/net/wlan1/queues/tx-3/xps_cpus

maurer · November 10, 2019, 12:48pm

considering that anyone can apply the speed fix via:

echo 3 > /sys/class/net/eth0/queues/rx-0/rps_cpus
echo 3 > /sys/class/net/eth0/queues/tx-0/xps_cpus

do you think it's still relevant that I compile these versions ?
please like this post if you do.
thanks

moeller0 · November 10, 2019, 4:47pm

To make these settings persistent on 18.06.X and 19.07 it might make sense to add

echo 3 > /sys/class/net/eth0/queues/rx-0/rps_cpus
echo 3 > /sys/class/net/eth0/queues/tx-0/xps_cpus

to the end of /etc/hotplug.d/net/20-smp-tune as otherwise a hotplug event on eth0 will reset the values away from 3.

Reiver · November 12, 2019, 2:51pm

Rather than adding the echo at the end, edit /etc/hotplug.d/net/20-smp-tune and change the set_hex_val parameters so all (non-virtual) interfaces are changed on hotplug.

for q in ${dev}/queues/rx-*; do
                set_hex_val "$q/rps_cpus" "$PROC_MASK"
                #set_hex_val "$q/rps_cpus" "$(($PROC_MASK & ~$irq_cpu_mask))"
        done

        ntxq="$(ls -d ${dev}/queues/tx-* | wc -l)"

        idx=$(($irq_cpu + 1))
        for q in ${dev}/queues/tx-*; do
                set_hex_val "$q/xps_cpus" "$PROC_MASK"
                #set_hex_val "$q/xps_cpus" "$((1 << $idx))"
                let "idx = idx + 1"
                [ "$idx" -ge "$NPROCS" ] && idx=0
        done

Though a configuration option to choose between;

all cores [$PROC_MASK]
irq handling core [0] (kernel default which disables packet steering)
non-irq handling cores [$(($PROC_MASK & ~$irq_cpu_mask))]
alternating cpu cores for multiple tx queues

Would be nice.

maurer · November 12, 2019, 3:04pm

have you tested this solution yourself?

Reiver · November 12, 2019, 3:17pm

Yes, tested and running (or ran since it's a hotplug script). It'd be hard to get this simple change wrong!

Although with git head, I'm getting an odd "No such file or directory" error when trying to read or write to xps_cpus files (that are present). Looks to be a kernel or selinux issue perhaps.

moeller0 · November 12, 2019, 7:16pm

I guess the real question is, whw are the defaults in 20_smp_tune as they are, and why do some routers/socs have issues, like the BTHH5A or the turris omnia. I think there is a common property of the problematic socs that might be detectable and might allow 20_smp_tune to do the right thing....

ParanoidZoid · November 13, 2019, 2:47am

Can't speak for the BTHH5A, but the patch below (which has been with mvebu devices since Kernel 4.4) could be one of the reasons why devices using mvneta + mvebu (as is the Turris Omnia) have quirky reactions to 20_smp_tune:

Which aims to fix this: https://bugs.openwrt.org/index.php?do=details&task_id=294

From Dave Täht 19.01.2017 22:57 in FS#294:

 Also, the current code is locked to the first core.

root@linksys-1200ac:/proc/irq# cd 37
root@linksys-1200ac:/proc/irq/37# echo 2 > smp_affinity
-ash: write error: I/O error
root@linksys-1200ac:/proc/irq/37# ls
affinity_hint node smp_affinity_list
mvneta smp_affinity spurious

Reiver · November 15, 2019, 2:31pm

I've created a pull request PR2553 to change the packet steering to use all CPUs but to also disable it by default. Hopefully the reasoning provided in the pull will be enough to be accepted. Essentially;

The non-irq CPUs for RPS just seems to come from just one Red Hat document without any actual performance testing.
The original packet steering patches advise that optimal settings for the CPU mask depend on architectures and cache hierarchy so one size does not fit all.
The original packet steering patches also advise that the overhead in processing for a lightly loaded server can cause performance degradation.
Proper IRQ balancing is a better option.

whiskerp · November 24, 2019, 5:28pm

I have tried running irqbalance and seen a 20% improvement in iperf3 results as well as greater consistency.