Netgear R7800 exploration (IPQ8065, QCA9984)

hnyman · November 16, 2016, 8:42pm

I got a nice suggestion from tohojo at the SQM scripts development repo, where I opened an issue about this. It may actually be how the HTB (hierarchical token bucket) qdisc perfoms on this CPU...

github.com/tohojo/sqm-scripts

ipq806x / Netgear R7800: difference in download speeds with cake and simple/fq_codel (possibly due to HTB )

opened 10:06AM - 16 Nov 16 UTC

closed 01:55PM - 30 Jan 17 UTC

hnyman

I have a new router Netgear R7800 that is an 1,7 GHz dual-core IPQ8065 device. I… have been building and testing new LEDE builds on it and I have stumbled upon strange behaviour with SQM using simple/fq_codel: * There is a significant speed gap between the set download speed limit and the realised download speed, as measured by flent. * With cake the same speed limit produces significantly higher download speed. * My old WNDR3800 (ar71xx) produces the same speed with cake and simple as R7800 with only cake. * Increasing the download speed limit by 10 Mb increases the actual download speed by maybe 7 Mbit/s, so the router is not hitting any real CPU or I/O performance limit. Data with wired LAN connection: * Without any SQM, my connection with R7800 produces 100 Mbit/s down, 12 Mb/s up, 35 ms latency * With speed limit down 90, up 8, the results are: * strategy / down / up / latency * R7800 simple: 67 / 7 /20 * R7800 cake: 84 / 7 / 19 * WNDR3800 simple: 83 / 5 / 18 * WNDR3800 cake: 83 / 5 / 19 * With speed limit down 100, up 8: * R7800 simple: 73 / 7 / 20 , So a 6-7 Mbit/s speed increase without negative impact on upload speed or latency. * With R7800 there was no significant difference between wired / wifi * I tried bumping kernel HZ from default 100 to 250, but that had no effect. It is almost like the simple/fq_codel would produce in ipq806x a realised download speed of some 75% of the set limit. With the old trustry ar71xx/WNDR3800 the speed gets higher upto the hardware limits. This almost looks like there is some kind of calculation bug in the fq_codel code when compiled for IPQ806x (code base is arm_cortex-a15_neon-vfpv4 ) Any clues to what could cause this? How to debug? * Can router's automatic CPU frequency scaling (between 384-1700 MHz according to load) cause trouble for codel? But not for cake?

Well in that case it sounds like it has something to do with the way HTB runs on that CPU. You could try if TBF has the same behaviour; enable sqm, then issue the following commands to replace the configured qdiscs with a TBF-based one (provided TBF is in LEDE; not sure if it is):

tc qdisc del dev eth0 root
tc qdisc del dev ifb4eth0 root
tc qdisc add dev eth0 handle 1: root tbf rate 8Mbit burst 15140 latency 100ms
tc qdisc add dev ifb4eth0 handle 1: root tbf rate 90Mbit burst 15140 latency 100ms
tc qdisc add dev eth0 handle 2: parent 1:1 fq_codel
tc qdisc add dev ifb4eth0 handle 2: parent 1:1 fq_codel

I tested that suggestion and did a quick and dirty Ookla speedtest and the resulting change is impressive:

simple with original HTB: 77 / 7 Mbit/s
simple with TBF (above): 85 / 8 Mbit/s

So, an immediate jump of 8 Mbit/s in the download speed and now the throughput is much closer to the set limit of 90 / 8. HTB might be culprit (but what is the ultimate reason for the bad performance, no idea, yet)