nods I can get behind that. Do you plan on trying to target power consumption? Packets per watt perhaps? ^-^
heh. Ideally I'd like to get away from mips. It's a dead end now, as much as I love the ath9k. I really haven't kept track of the market (?) and hope that mt76 stuff was dropping down in price. The gl300 was a pretty good box...
To the 10Gbit point, showing how cake can improve a 10Gbit link - on some cheap platform - is also on my mind. I mostly fear we have to make that big move to xdp and a xpf firewall compiler to make progress at these speeds.
The mikrotik folk have finally added fq_codel and cake across their product line in the beta, but there's no means to dynamically reconfigure it. They do make some interesting looking hw that can be reflashed to openwrt.
I would really love a proof of concept that used libreqos on openwrt, on a wndr3800.
2 Likes
While I am doing the blue sky thing, NDN still lives: https://named-data.net/publications/ndn-hackathons/
We have fq_codel support in our community nss builds for the IPQ806x, which is accelerated by the nss cores.There is pretty much no cpu load when shaping 1Gbps. I added a very basic script for that, in order to be selectable in webif:
It could use some work, but I don't have much time lately.
It's also worth asking if there is something simpler and lower computation than cake which could help these speeds. Like hashing to bfifos below a qfq, and a TBF on top maybe. Even 1000 MTU packets takes 1.2ms at 10G speed so you could have a lot of slop compared to optimal without much bloat.
2 Likes
Well I used flent to check if it is working, it is not as good as cake but it is ok.
@kong thx for testing with flent. Got the flent.gz files anywhere? Flent mostly shows fq working not codel working unless you stare at it hard.
Also the burst parameter seems to be hard coded very, very large.
A good test is to knock it to 200Mbit and use
--step-size=.05 --socket-stats --te=upload_streams=4 tcp_nup
For science! I have a nearly religious love of drop head queuing vs tail drop, but showing any benefits and 10gbit performance on cheap hardware of anything but a FIFO would be good.
Unfortunately I deleted them, but I can run the tests again tomorrow. But I only have a 100Mbps ISP connection, for higher speeds I would have to setup some local server on the wan side of the router.
Not sure if that comes close to a real world scenario.
well, my concern is that there's problems above 100Mbit. Does the nsstbf work at 900mbit? inbound?? It would be cool if it worked in bound.... It's a very real world scenario to have a server "right there" on the wan side....
But: Certainly verifying that it worked at 100Mbit would be a start! But the regular sqm-scripts use a scalable burst and htb quantum you don't have....
Exactly. The CPU requirements at 10Gbps are significant. Something computationally cheap enough to run with routing and firewall on say a Celeron, particularly if it could be multithreaded, would be a lot better than a 1000 element pfifo.
So how dead IS mips, anyway. There's a new ebpf jit for it now: https://lwn.net/Articles/874683/
1 Like
I have a spare R7800 not (yet) actively in use and a fiber connection of 500/500mbps. I’ve never done anything with flent, but if you guys need some testing on speeds above 100mbps and can “talk me through it”, just let me know.
Start by installing flent from the flent.org site or via apt/yum/etc.
My desired topology is client - router - server - nothing else, so we can merely verify correctness. Do you have a decent box to use as a server?
OK, here is the first test:
flent --step-size=.05 --socket-stats --te=upload_streams=4 tcp_nup -H netperf-eu.bufferbloat.net
flent --step-size=.05 --socket-stats --te=download_streams=4 tcp_ndown -H netperf-eu.bufferbloat.net
ISP speed is ~100Mbps/35Mbps. I have configured 95000 down and 30000 up.
--socket-stats only works on the up. Both the up and down look ok, however the distribution of tcp RTT looks a bit off. These lines should be pretty identical.
Three possible causes: 1) your overlarge burst parameter. At 35Mbit you shouldn't need more than 8k!! The last flow has started up late and doesn't quite get back into fairness with the others. 2) They aren't using a DRR++ scheduler, but DRR. 3) Unknown. I always leave a spot in there for the unknown and without a 35ms rtt path to compare this against I just go and ask you for more data
No need for more down tests at the moment.
A) try 8 and 16 streams on the up.
B) try a vastly reduced htb burst.
A packet capture of a simple 1 stream test also helps me on both up and down, if you are in a position to take one. I don't need a long one (use -l 20) and tcpdump -i the_interface -s 128 -w whatever.cap on either the server or client will suffice.
I am temporarily relieved, it's just that the drop scheduler in the paper was too agressive above 100mbit... and we haven't tested that on this hardware....
Anyway this more clearly shows two of the flows are "stuck":
