CAKE w/ Adaptive Bandwidth [August 2022 to March 2024]

i wonder if i can ask chatgpt to re right this as if i was 10 lol so i can understand im gonna mess with it and see where i like it

im just trying to get lowest latency for fps gaming while everyone els still uses the internet.

I am sorry if this is too convoluted, maybe ask specific questions?

So Minimum=~? (before its too low and can't recover~ from latency being too high due to lack of bandwidth)

Base = Stable bandwidth you expect 70%+ of the time~

Max= Unstable bandwidth but link is capable of getting near
(so when the script detects downloads/streaming it basically ramps towards the max essentially~ ignore the delay-ish)

As per your suggest I've greatly improved my throughout put and latency control, you guys have been amazing.~

as far as i've experienced the ext4 firmware appears to actually work with the script as the squashfs~ appears to hate the living crap out of me as it wasn't working without reinstalling the script every few days...

However it may seem outside of my control it appears that when my download drops to 128kbps during peak the script does not recover. maybe due to isp interference??

(i think my isp may be doing something else to "their" network I'm not a fan of the way ipv6 has been implemented as its... unnecessarily complicated and not exactly compatible for the World Wide Web---since I have no... "manageable" public ipv6)

how can you minimize the cpu usage with this script for a lower end device if it is possible ?

It's always a combination of your WAN's maximum speed and the router's performance you throw at it, ideally the router is always fast enough to cope with sqm/cake at your maximum WAN speed (plus a healthy margin). If your router is slower than (too slow for) that, you'll have to reduce the max value to a limit the router can still deal with comfortably (otherwise the router itself will add delays, as it's overloaded). The approach to find this sweet spot, would be to just try (keeping an eye on htop and the speedtest/ bufferbloat results) during the testing - start with a high maximum, reduce as much as needed (and a little less than that). You don't need this script for those tests, plain sqm/cake (as it's less dynamic, less of a moving target) and its limits are probably quicker at finding the limits of your device (your findings should still apply to autorate as well).

As moeller0 implied, the end result is always a policy decision, your own - how far you want to go, how much (potential) WAN speed to sacrifice, not because of the ISP being bad (which is bad enough, but rarely something you can change, or you would have already), but 'just' because the router is at its performance limits (something you can change, with faster router hardware).

but the problem is that even when im not generating any traffic whatsoever the cpu is still maxed out
i understand that cake in it self is cpu demanding compared to other shapers , but in this situation it's the script that is maxing out the cpu without any traffic

Yes, that script is requiring some CPU as well, how much, mostly depends on the effective frequency of checking reflectors...but so does the "responsiveness" of the script. One solution is to simply use performant enough hardware for the desired sampling frequency, but that is not helpful for users that already have an otherwise well-working OpenWrt router....

1 Like

i made a fork with a single commit, using fq_codel/simplest_tbf instead of cake, for those needing lower cpu usage, feel free to try

4 Likes

Howdy all! Hoping to get some help in possibly setting cake up properly for my connection. Not sure what I'm doing wrong, but no matter what I do, my bufferbloat situation is bad on the download side. Enabling cake on the egress side only, the upload bufferbloat is extremely minimal (+5-10ms as opposed to +300 or more without it) yet the download bufferbloat stays the same (+100 and higher) and most times worsens when enabling cake (and autorate as well). I've had a select few times where it's decreased the to +40-50 but that's been rare. I'm mainly concerned with getting latency as stable as possible for gaming.

I'm on AT&T's FirstNet, feeding a Pi 4 which feeds another OpenWRT device as a dumb AP. I average 50-60 megabits down, 12 up on most given days, sometimes maxing at 110, never really anything below 45. I feel as if I am missing something when configuring this but not exactly sure what. I'm no network administrator but I know a few things, this much is beyond me though. I'd appreciate any help anyone has!

I would propose we start simple.... what do you use to initially configure cake, cake-qos-simple or sqm?
Also could you post the output of:

  1. tc -s qdisc
  2. ifstatus wan | grep device
    Also I would like to see result links/screenshots for the following throughput-test (commonly misnamed as "speed-tests", but speed is distance/time, which is not what these tests measure):
    a) disable traffic shaping/cake-autorate completely:
    b) tc -s qdisc
    c) https://www.waveform.com/tools/bufferbloat
    d) https://speed.cloudflare.com

e) re-enable traffic-shaping/cake-autorate
f) tc -s qdisc
g) https://www.waveform.com/tools/bufferbloat
h) https://speed.cloudflare.com
i) tc -s qdisc

2 Likes

Thank you so much for the quick reply! I configured cake very shoddily with SQM in Luci. It's quite possible I don't have that part configured properly. And of course, my hotspot/modem is locking onto Band 14 right now which is a little better on latency and worse on speed than the usual Band 2 which I get higher speeds and wilder latencies/bufferbloat. Sorry for the linked images for cloudflare tests, as a new user it only would let me embed one :confused:

tc -s qdisc (cake-autorate off)

qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc mq 0: dev eth0 root
 Sent 359614 bytes 1425 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth0 parent :5 limit 10240p flows 1024 quantum 1356 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 13432 bytes 118 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :4 limit 10240p flows 1024 quantum 1356 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 116831 bytes 356 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :3 limit 10240p flows 1024 quantum 1356 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 66918 bytes 199 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :2 limit 10240p flows 1024 quantum 1356 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 88856 bytes 258 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1356 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 73577 bytes 494 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 15315820570 bytes 12612487 pkt (dropped 0, overlimits 0 requeues 152)
 backlog 0b 0p requeues 152
  maxpacket 16272 drop_overlimit 0 new_flow_count 4639 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev wlan0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 14874328 bytes 68443 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 1512 drop_overlimit 0 new_flow_count 1124 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br-lan root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
root@OpenWrt:~/cake-autorate# tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc mq 0: dev eth0 root
 Sent 461634 bytes 2037 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth0 parent :5 limit 10240p flows 1024 quantum 1356 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 29703 bytes 270 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :4 limit 10240p flows 1024 quantum 1356 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 172923 bytes 588 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :3 limit 10240p flows 1024 quantum 1356 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 82442 bytes 294 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :2 limit 10240p flows 1024 quantum 1356 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 96671 bytes 323 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1356 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 79895 bytes 562 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 15316156816 bytes 12613109 pkt (dropped 0, overlimits 0 requeues 152)
 backlog 0b 0p requeues 152
  maxpacket 16272 drop_overlimit 0 new_flow_count 4639 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev wlan0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 14895754 bytes 68512 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 1512 drop_overlimit 0 new_flow_count 1124 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br-lan root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0

tc -s qdisc (with cake-autorate on)

qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 806f: dev eth0 root refcnt 6 bandwidth 2463Kbit besteffort dual-srchost nat nowash no-ack-filter split-gso rtt 100ms noatm overhead 44 mpu 96
 Sent 26809725 bytes 116051 pkt (dropped 3002, overlimits 178115 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 242348b of 4Mb
 capacity estimate: 7Mbit
 min/max network layer size:           28 /    1342
 min/max overhead-adjusted size:       96 /    1386
 average network hdr offset:           14

                  Tin 0
  thresh       2463Kbit
  target         6.61ms
  interval        102ms
  pk_delay        450us
  av_delay         42us
  sp_delay          3us
  backlog            0b
  pkts           119053
  bytes        29221326
  way_inds            3
  way_miss          250
  way_cols            0
  drops            3002
  marks               0
  ack_drop            0
  sp_flows            4
  bk_flows            1
  un_flows            0
  max_len         13560
  quantum           300

qdisc ingress ffff: dev eth0 parent ffff:fff1 ----------------
 Sent 123373419 bytes 113567 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth1 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 16186715285 bytes 13378412 pkt (dropped 0, overlimits 0 requeues 810)
 backlog 0b 0p requeues 810
  maxpacket 58308 drop_overlimit 0 new_flow_count 9756 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev wlan0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 15302259 bytes 69812 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 1512 drop_overlimit 0 new_flow_count 1130 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br-lan root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8070: dev ifb4eth0 root refcnt 2 bandwidth 25555Kbit besteffort dual-dsthost nat wash no-ack-filter split-gso rtt 100ms noatm overhead 44 mpu 96
 Sent 124853077 bytes 113403 pkt (dropped 164, overlimits 180912 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 505730b of 4Mb
 capacity estimate: 40Mbit
 min/max network layer size:           46 /    1342
 min/max overhead-adjusted size:       96 /    1386
 average network hdr offset:           14

                  Tin 0
  thresh      25555Kbit
  target            5ms
  interval        100ms
  pk_delay        935us
  av_delay        115us
  sp_delay          4us
  backlog            0b
  pkts           113567
  bytes       125074381
  way_inds            7
  way_miss          243
  way_cols            0
  drops             164
  marks               0
  ack_drop            0
  sp_flows            3
  bk_flows            1
  un_flows            0
  max_len         13560
  quantum           779

ifstatus WAN | grep device

"l3_device": "eth0",
        "device": "eth0",

Waveform test, no cake: https://www.waveform.com/tools/bufferbloat?test-id=da5334c9-2278-4b7e-a47c-dd2f0f04ea01

Cloudflare, no cake: https://imgur.com/a/PH7hXvO

Waveform test, with cake-autorate: https://www.waveform.com/tools/bufferbloat?test-id=4d57c4cf-b869-47d2-ab47-2fb4697615b0

Cloudflare, with cake-autorate: https://imgur.com/a/iW5e1Kx

My autorate config (no overrides, tried playing with them a few times but not sure exactly what I'm tweaking with it):

dl_if=ifb4eth0 # download interface
ul_if=eth0     # upload interface

# Set either of the below to 0 to adjust one direction only
# or alternatively set both to 0 to simply use cake-autorate to monitor a connection
adjust_dl_shaper_rate=1 # enable (1) or disable (0) actually changing the dl shaper rate
adjust_ul_shaper_rate=1 # enable (1) or disable (0) actually changing the ul shaper rate

min_dl_shaper_rate_kbps=5000  # minimum bandwidth for download (Kbit/s)
base_dl_shaper_rate_kbps=15000 # steady state bandwidth for download (Kbit/s)
max_dl_shaper_rate_kbps=40000  # maximum bandwidth for download (Kbit/s)

min_ul_shaper_rate_kbps=500  # minimum bandwidth for upload (Kbit/s)
base_ul_shaper_rate_kbps=1000 # steady state bandwidth for upload (KBit/s)
max_ul_shaper_rate_kbps=5000  # maximum bandwidth for upload (Kbit/s)

My SQM configuration, if it's any help (saw 44 mentioned as a safe overhead, also FirstNet has a strange MTU of 1328, so I changed max packet size from default of 2047 to 1875 as I had assumed that that number itself assumes an MTU of 1500, so I figured out the difference and changed it just screwing around trying things lol)
https://imgur.com/a/SoJTmVK

That really is not correct, or rather not aligned how these numbers are interpreted by tc's stab option, however cake does its own atm accounting and ignores these numbers and your link does not use atm in the first place...

If your gaming spikes came without cross traffic from the rest of your network and also occur without cake running, these might not be fixable by a local traffic shaper....
EDIT: I mixed up two threads and thought your challenge was gaming.... will have a closer look later today and hopefully come up with a more meaninful reply....

For cake-autorate tests, the best you can do is to export a log file directly after your throughput tests so we see how throughput and latency develops over time... as well as what shaper settings the script selected.

These are a bit tricky... at 500 Kbps a single full MTU sized packet (depending on the used encapsulation around 1540 bytes in size, but let's just run the numbers for 1500) will take:

[milliseconds/second] * ([bytes] * [bits/byte]) / ([Kilobit/second] * [bit/Kilobit]) = [milliseconds]
1000 * (1500 * 8) / (500 * 1000^1) = 24 ms

That means on average your upload delay under load will be around 12ms and the jitter quite large. At those speeds, all we can do is to manage the pain.... and at 1Mbps the per packet transmission time is still 12 milliseconds... autorate will adjust the delay threshold so the shaper will take this into account, but your experiences delay will be higher by necessity. If you are lucky the minimal upload rate rarely triggers (but we need to look at your autorate logs to confirm that) but the base_ul rate will be where the controller starts after some idle time, and 1Mbps is still in the "manage the pain region".

What are the delay thresholds you configured for autorate?

1 Like

If I'm understanding the question correctly, I have the thresholds at the default 30ms on both upload and download

Maybe you could post your full config file again? As well as a log (@Lynx could you remind us on the exact invocation to extract the log with the current code? I have not been running any recent autorate, so do not know how this is done ATM...)

Hi,

I already use cake with piece of cake script and the luci app.
Installing your script will allow me to just select autorate under the same luci app?
Or should I remove anything cake related before installing your script?

Thanks.

No, cake-autorate is a different beast, however it is compatible with sqm/piece_of_cake in that the autorate script expects you to somehow instantiate your initial cake qdiscs, which is what sqm can deliver easily.

You will need something or need to edit the script to start the required cake instances.

Hi @moeller0
Thank you for your reply.

I went through the github page and had a quick read.
It seems that maybe for me it won't do much. I have FTTH connection so I always have a stable bandwidth from my ISP.

Also already with piece of cake I get A+
So I don't believe it will dustify the extra cpu load.
Especially since I already run a schedule to automatically do speedtest and re-apply the SQM values everyday...

That been said, this is a good solution and I wish I had it at the before FTTH years!

2 Likes

I am in a similar boat, my dsl connection is refreshingly boring and always delivers maximal contracted rates (fullsync), all autorate can do on such a link is 'not getting in the way'...
Now on a variable link rate from a less customer friendly ISP things can look quite differently...

1 Like

** cries in FTTH with highly variable several unstable isp-**

That's very fortunate, I'd imagine that cake-autorate Could~ at the very least improve the micro seconds uS instead of ms.

if you live near the servers you're doing the speed tests, everything will essentially be A+ regardless, at that point you wanna look and see if the jitter varies, and just assume if theres high jitter on the network there might be an underlying issue that the isp should've resolved.~~ autorate might be able to improve~

I've seen cake-autorate being able to lower my jitter and overall bloat significantly, and it noticeably impacts the response and fluidity of online games, now the only people that teleport are the ones with high ping and not everyone
(I don't remember this being an issue in like 90% of games 10-15 years ago but here we are)

( we shouldn't have to worry about this normally unless the whole countries networks get overloaded during certain times.... where I currently reside.. it seems that there is alooooot of catching up to do when it comes to stability and quality... speed isn't everything.... especially if you get long duration 3000ms+ spikes while on a 2.5gbps line :expressionless: )

The autorate works very efficiently now and I struggled setting it up initially So I'd like to recommended it to almost everyone but not everyone has the resources to try it out sadly.

I'm looking forward to the projects This community puts together its super interesting everything they're doing and improving