Hi all,
i'm wondering, is it normal on a BT HomeHub 5A doing PPPoE + NAT + routing + firewall + SQM/cake, at ~80 Mbps and SFO, to measure 0% cpu usage during normal (domestic) network traffic (with 2-4 devices) and even during more demanding speed tests? Is top lying? Ah, i have no wifi enabled on the router by now...
Honestly i'm quite surprised, as I was reading around the forum that hardware in this range could barely route 200Mbps and absolutely no SQM nor PPPoE, so i was expecting some cpu load at least... maybe those were the expectations without SFO?
Not really - or perhaps, depending on what you're testing.
Traffic between different wired LAN stations is handled exclusively in the internal managed hardware switch and won't be seen by the router (and therefore can't tax it).
Routing to the internet at 80 MBit/s (and even more with SQM enabled) however will push the bthub5 to its limits. The bthub5 can do around 84 MBit/s without software flow-offloading or SQM enabled. If you enable software flow-offloading, you can usually push that towards ~110-120 MBit/s - but this clashes with SQM. Enabling SQM will require considerably more power from your router (and as a side-effect kills the benefits of software flow-offloading, so you can just as well keep it disabled), as a consequence I don't expect you to (still) achieve 80 MBit/s throughput while running SQM on this hardware (more like ~55-60 MBit/s at most).
Just for a comparison:
VDSL2+vectoring (profile 17b), the bthub5 is only acting as ethernet router - not using its internal modem (due to wiring difficulties at the moment)
running a speed test will push one core to >95% (the other remains at ~30-35%, routing traffic is mostly single-threaded, leaving the second core dormant), htop is better at visualizing this than top. Actually using the VDSL modem will not change these figures that much (the modem functionality is mostly handled in hardware/ firmware), heavy WLAN traffic is rather prominent in the CPU usage as well.
Wow @slh thanks for your exhaustive answer... and in some way you anticipated me on one more thing i was going to add...
Ah sure, i was talking about LAN/WAN, sorry if it was not clear enough...
Indeed, what i was going to point out is that i noticed that with SQM i was hitting ~55Mbps max, even if the shape rate was set to 76Mbps (95% of 80Mbps, effective rate out of 100Mbit plan). But since i'm new to SQM and its parameters tuning, i didn't know wheter this was an expected behaviour or not, also considering that i was seeing a steady 0% cpu usage in top, and i didn't think it was due to the extreme load.
So yeah, i was meaning it was shaping an 80Mbps bandwidth, not routing, which isn't entirely true either, since it maxes out at ~55Mbps as you pronosticated.
Ah here's the catch... maye i should have some tests with htop, even if it looks like you unintentionally did all the tests for me (almost same usage/line conditions).
What misleaded me is that i can see cpu load variations when using e.g. LUcI interface or for top itself, so i was assuming top was working as expected... (now i would also be curious why top is not showing that kind of load, but maybe it's OT)
Ok i guess this must be about top not considering kernel threads? with htop i see userspace threads that account for the load of one core, while the other core goes 100% because of ksoftirqd (using shift+k to see it). Didn't remember there was such difference between top and htop...
top does display those, but it makes it a little harder to interpret than htop - as multi-core devices aren't properly displayed. 50% CPU usage on the bthub5 (dual-core) means 100% load on one core, while htop will display them individually. The bthub5 is a great device, but not very fast - nor fast enough for sqm above ~50 MBit/s.
If you want to use sqm on your connection, relegating the bthub5 to mere (bridged-) modem duties and terminating the PPPoE session on a more capable router would be a better idea.
Software flow offload does work fine with SQM only hardware offload doesn't. There is another similar thread where that was hashed out and tested the last few days.
I intentionally phrased it as "kills the benefits of software flow-offloading" and not as "doesn't work", as you lose most of the speed-up generally associated with software flow-offloading, although it technically still works. There just isn't much to offload anymore, as SQM needs to treat packages individually - making them take the normal/ non-offloaded ('slow') path through netfilter, instead of being able to recognize (much of-) a flow and treating them as whole (~offloading). It is safe to enable software flow-offloading and sqm at the same time, but you won't see much of a speed difference with- or without software flow-offloading while sqm is actively working.
No I don't think that's right. The SFO still sends packets through the fast path, but then after the fast path SQM does still have to look at them all and it uses a lot of cpu. So you do save all those iptables related cycles but you still have to spend the same number of SQM cycles.
The bthub5 (with SMP enabled) can achieve around 82-84 MBit/s without software flow-offloading - and around (just barely) 110-120 MBit/s by using software flow-offloading. At ~80 MBit/s WAN speed of the OP's VDSL connection, it's already running with the back to the wall. Regardless of software flow-offloading being enabled or disabled (or being effective in the presence of sqm), this hardware can't do sqm at the required "76Mbps (95% of 80Mbps, effective rate out of 100Mbit plan)".
Ah sure top shows the overall usage across all cores... the problem is i was just trying to spot the current usage in the usr column, and it was around 0% all the time, while the sirq column (damn, the last one!) was in fact going around 50. As an most-of-the-time-htop-user myself, i totally overlooked that, my bad...
Well since i have the ISP modem that does VoIP too and sends it to the house phone line, i can't immediately replace it with the BT. Also having another more powerful router is a bit too much for the typical uasage/users of this network... Hopefully i will upgrade when I can find a good deal.
For now, to keep using the BT as router for the home network - which is what i care about mostly - without giving away bandwidth (and also keeping some room for e.g. wifi), i've just disabled SQM and doing double routing into the ISP modem, taking advantage of some sort of builtin QoS on that box (that also takes care of the PPPoE connection now).
It's not the same thing as having SQM, but from some tests it's still better than simply disabling SQM, and using that unregulated bandwidth. It's somehow in a middle spot:
Mode PPPoE SQM Latency Load Mbps
---- ----- --- ------- ---- ----
Double routing no no up to +50 ~35% 80+/20+
Bridged ISP modem yes no up to +100 ~100% 80+/20+
Bridged ISP modem yes yes up to +15 ~100% ~55/20+
--
Latency: relative to the idle latency to 8.8.8.8 (ping)
Load: of the 'routing' core
Tests may not be very accurate but you get the idea... (If i feel like i will do better ones on dslreports)