CAKE w/ Adaptive Bandwidth [August 2022 to March 2024]

With -c 100 it will take north of 100 seconds to return something (if you leave out the -c 100 it will run interactively showing intermediate results almost immediately).

This is during a stable~ connection, the unstable connections are completely variable, or whenever the ISP decides to give you the middle finger

HOST: Fangler                                                                 Loss%   Snt   Last   Avg  Best  Wrst StDev 
1. ASN<- cloud   reduced caboose)                                            0.0%    10    2.5   2.3   1.9   3.1   0.3  
2. AS???    ???                                                             100.0    10    0.0   0.0   0.0   0.0   0.0  
3. AS???    ???                                                             100.0    10    0.0   0.0   0.0   0.0   0.0  
4. AS???    ???                                                             100.0    10    0.0   0.0   0.0   0.0   0.0  
5. AS4713   2001:380:c510:b::1 (2001:380:c510:b::1)                         10.0%    10   21.3  21.1  18.8  29.8   3.5  
6. AS4713   2001:380:c500:25::1 (2001:380:c500:25::1)                        0.0%    10   19.3  19.7  18.8  22.5   1.2  
7. AS4713   2001:380:c200:7::1 (2001:380:c200:7::1)                          0.0%    10   27.6  27.4  27.1  27.6   0.2       
[MPLS: Lbl 3954 TC 0 S u TTL 1]
[MPLS: Lbl 2 TC 0 S u TTL 1]
8. AS4713   2001:380:c200:8::2 (2001:380:c200:8::2)                          0.0%    10   28.3  31.8  27.3  52.0   7.5  
9. AS2914   ae-12.r02.osakjp02.jp.bb.gin.ntt.net (2001:218:2000:5000::929)   0.0%    10   27.4  28.0  27.1  32.3   1.5 
10. AS2914   2001:218:2000:5000::162 (2001:218:2000:5000::162)                0.0%    10   27.6  31.5  27.6  43.3   6.0 
11. AS13335  2400:cb00:370:3:: (2400:cb00:370:3::)                            0.0%    10   54.5  31.1  27.5  54.5   8.3 
12. AS13335  2606:4700::6810:4426 (2606:4700::6810:4426)                      0.0%    10   27.3  29.1  27.3  33.5   1.9

I uhh... I'll try to fix the above later.
This is what it spat out~

2 Likes

Great, this is with reasonably high throughput and decent latency, will be interesting to compare this with data from peak-hour when the latency doubles...

mmmm It may take a few days for the similar situation, I honestly don't know what causes it but eventually the network will just suffer for no apparent/reasonable reason... for a couple of days..

Hi, I have difficulties to set up cake-autorate/SQM adequately. I deactivated cake-autorate and SQM, went to waveform.com for a RTT test and also used gping, which gave me roughly the same data:

I set the thresholds for cake-autorate as following;

  • min_dl_shaper_rate_kbps=5000
  • base_dl_shaper_rate_kbps=75000
  • max_dl_shaper_rate_kbps=88000

After launching cake-autorate and activate SQM (with the following settings)....

...It seems it does not really much to reduce bufferbloat

Please post your full config file and a link to the autorate log preferably containing a speedtest and a link/screenshot of that speedtest's results...

Ensure the openwrt image is EXT4 as the squashFS doesnt seem compatible (for me at least~)

Also make sure the cake-autorate service is enabled, and you type start~

All these below will be involved in the SSH method through either WindowsPowerShell, or in a linux equivalent Shell~ (ex. ssh root@192.168.1.1 <- default openwrt ssh)

service <- this alone will show all services, try to see if cake-autorate is listed and enabled

service cake-autorate enable
(there is enabled and enable, enable will enable it to load on start up)

service cake-autorate start
(will start the actual service on startup)

service cake-autorate info
(just to see if the service has properly loaded, asfaik it will start on squash but unable to actually work~)

@moeller0 i'm confident I'll be able to get the mtr results by tomorrow/today, I'm 100% positive that its more apparent on the weekends.... Though I am curious why my latency literally doubled~ I don't Think my ISP is incompetent.. just trying too many new protocols with new methods that... isn't really efficient but... I want to blame the cellular network lacking ipv6 as the primary connection method for a majority of some weird wonky bonanza issues

                                                                               Loss%   Snt   Last   Avg  Best  Wrst StDev
  1. AS-   2400:-(2400:-)                                                   0.0%    10    2.0   2.7   2.0   3.6   0.5
  2. AS???    ???                                                             100.0    10    0.0   0.0   0.0   0.0   0.0
  3. AS???    ???                                                             100.0    10    0.0   0.0   0.0   0.0   0.0
  4. AS???    ???                                                             100.0    10    0.0   0.0   0.0   0.0   0.0
  5. AS4713   2001:380:c510:16::1 (2001:380:c510:16::1)                        0.0%    10   19.0  21.7  18.3  25.1   2.6
  6. AS4713   2001:380:c500:26::1 (2001:380:c500:26::1)                        0.0%    10   30.8  24.3  19.8  30.8   4.4
  7. AS4713   2001:380:c300:6::1 (2001:380:c300:6::1)                          0.0%    10   33.7  37.8  30.6  64.1  10.0
       [MPLS: Lbl 4847 TC 0 S u TTL 1]
       [MPLS: Lbl 2 TC 0 S u TTL 1]
  8. AS4713   2001:380:c300:7::2 (2001:380:c300:7::2)                          0.0%    10   33.8  47.2  33.7 126.0  28.8
  9. AS2914   ae-11.r02.osakjp02.jp.bb.gin.ntt.net (2001:218:2000:5000::931)   0.0%    10   42.5  36.9  33.4  48.2   5.0
 10. AS2914   2001:218:2000:5000::162 (2001:218:2000:5000::162)                0.0%    10   49.2  51.3  34.3  96.7  22.5
 11. AS13335  2400:cb00:51:3:: (2400:cb00:51:3::)                              0.0%    10   45.3  47.6  33.8  73.4  14.0
 12. AS13335  2606:4700::6810:4326 (2606:4700::6810:4326)                      0.0%    10   39.2  36.5  33.3  47.6   4.4


along with the speed test, idk whats doubling my ping, hopefully this makes it a bit more clear :confused:

Along with a Waveform test~

HOST: Fangler                                                                 Loss%   Snt   Last   Avg  Best  Wrst StDev
  1. AS-   2400:-                                                 )           80.0%    10    2.8   2.7   2.7   2.8   0.1
  2. AS???    ???                                                             100.0    10    0.0   0.0   0.0   0.0   0.0
  3. AS???    ???                                                             100.0    10    0.0   0.0   0.0   0.0   0.0
  4. AS???    ???                                                             100.0    10    0.0   0.0   0.0   0.0   0.0
  5. AS4713   2001:380:c510:16::1 (2001:380:c510:16::1)                        0.0%    10   21.3  24.3  19.4  32.4   4.4
  6. AS4713   2001:380:c500:26::1 (2001:380:c500:26::1)                        0.0%    10   23.2  25.4  19.6  38.7   5.8
  7. AS4713   2001:380:c300:6::1 (2001:380:c300:6::1)                          0.0%    10   31.7  38.4  31.4  57.7   9.2
       [MPLS: Lbl 4847 TC 0 S u TTL 1]
       [MPLS: Lbl 2 TC 0 S u TTL 1]
  8. AS4713   2001:380:c300:7::2 (2001:380:c300:7::2)                          0.0%    10   43.2  40.6  33.4  49.0   5.1
  9. AS2914   ae-11.r02.osakjp02.jp.bb.gin.ntt.net (2001:218:2000:5000::931)   0.0%    10   39.4  39.0  33.8  44.6   3.8
 10. AS2914   2001:218:2000:5000::162 (2001:218:2000:5000::162)                0.0%    10   49.8  42.4  34.3  49.8   5.1
 11. AS13335  2400:cb00:51:3:: (2400:cb00:51:3::)                             10.0%    10   45.9  50.9  33.7  73.2  10.5
 12. AS13335  2606:4700::6810:4326 (2606:4700::6810:4326)                      0.0%    10   48.2  39.9  33.3  49.3   5.6

AS13335: NTT Communications Corporation (OCN)
AS2914: NTT Global IP Network
AS13335: Cloudflare

~20ms to the first reliably responding hop, and the addition al ~10ms, likely for the MPLS tunnel, all within your ISPs own network (you are NTT customer, no?) More or less similar to your earlier MTR results.

I am probably a bit daft today, but where do you see the doubling of the ping? (I believe your assessment, I just can not see where in the data, and having stared long enough, I am sure today is a day for asking, not for staring even longer :wink: )

Latency will on a normal day should be from 28ms-30ms ingames, however when my internet does similar things to the above post my ping will be doubled well either way.. 45ms-55ms isn't normal (latency is doubled on ingress, as the mean is 88ms and thats the ping i'd get in games)

(60ms~80ms not jittering up to 60ms-80ms and going back down, but staying at 60ms-80ms)

I'm not sure if this is due to the loss% tbh but its not... a pleasant experience especially since it lasts 3-4 hours and this is for everyone in the country outside of tokyo/osaka asfaik.

These "issues" are entirely new for me as.... when I was in the states the latency was stable all the time and only improved with cake-autorate. (i had 80ms~bit far from the.. mainland then with cake it was down to 60ms on a 300mbps using a docsis 3.1 isp provided modem on the line )

However using cake-autorate in japan can improve the jitter ~
All i've noticed is that.... sadly the internet feels completely unstable with and without cake... and I can't figure out why.. as my isp doesn't really want to communicate/unable to?

Mmmh, we need a OWD test for your network to compare good and bad times to see whether both OWDs increase or only in one direction...

@moeller0 I had a good look just now throughout this forum for discussions covering this topic. I saw a few simplistic tests trying to compare 'ondemand' and 'performance', but those seemed inconclusive.

Have you encountered evidence of frequency scaling hurting performance in cake?

And, assuming there is a performance impact, I wonder how we might characterize the impact in terms of form and degree?

I am supposing a periodic 1-5ms difference might make much more of a difference for low-latency fixed rate connections with a low average latency (say circa 5-10ms to begin with), whereas 1-5ms difference might not be so significant for higher-latency connections like 4G with an average latency of 40-50ms and goal to keep bloat below 100-200ms. But I am speculating here.

And could there be any special considerations relating to adapting the bandwidth in cake in the way we do in cake-autorate?

In short, I am trying to work out whether to switch to 'performance' rather than 'ondemand' frequency scaling for my main router, but I am also interesting in the theory and any concrete tests or evidence surrounding this topic.


By the way, cake-autorate just seems to be working very well in general now - I haven't changed the code for some time.

Personally no, but then my turris omnia always runs at a fixed frequency (it does not allow frequency scaling IIRC). But this has come up in the forums for other users. Theoretically it seems plausible, cake needs CPU with low latency when it needs it, and if it gets the CPU too late both throughput and latency will suffer. It seems that the cake load is too bursty for some governors, but then this is second hand experience.

Well, what I would try to do is, add logger functions to cake-autorate that measure and log the frequency for each CPU as well as the load per CPU (like 100-%idle). Then if these are added to the autorate plots we should be able to see whether we get reliable latency spikes at times when the frequency changes and also see how such changes affect CPU load.

Only in so far as, if cake runs out of timely CPU cycles autorate will throttle the shaper down, likely into a regime where cake gets enough CPU, so in a sense cake-autorate already implements a work-around for CPU overload :wink:

Good question, I guess the proof is in the pudding (or given the time of year "in the plum pudding"). I guess a cheapskates test would be to run cake-autorate for a few days with either governor, while recording log files, and then somehow compare the total aggregated amount of (delta) delay between the two (for epochs with noticeable load?). Not sure though whether we are prepared for long term logging.

Sometimes "boringly" robust and reliable is good, no? :wink:

This is one of the best projects I've come across, optimizing our countryside house to the best of its ability.

FWIW, networking equipment as such, I've personally always turned off from using power management sleep functions. In other words, perf. governor where applicable C states off etc.
It's anecdotal but I say it makes a difference, rarely but still, noticeable.

I run it on an old PC from a decade ago, it's literally overkill but it serves me very well for what it was in the past, and selling it for scrap buck is just a waste IMHO. Runs an i7-3770k. x86_64

kernel command line has these appended:
intel_idle.max_cstate=0 processor.max_cstate=1

intel_idle disables the intel driver where it's compiled in. The other one defaults the acpi PM to C1 as the max sleep state.

in rc.local I have:
echo performance | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

SO

1 Like

Well, if you have a latency of less than 1 ms to begin with, the CPU speed variation's impact on calculations could be really visible.
I have fiber-to-the-home, so the latency to the local large sites in Finland is really low. E.g. two largest local media companies are 0.55 ms and 0.58 ms on average...

root@router5:~# ping -c 4 www.hs.fi
PING www.hs.fi (108.156.22.27): 56 data bytes
64 bytes from 108.156.22.27: seq=0 ttl=250 time=0.594 ms
64 bytes from 108.156.22.27: seq=1 ttl=250 time=0.548 ms
64 bytes from 108.156.22.27: seq=2 ttl=250 time=0.536 ms
64 bytes from 108.156.22.27: seq=3 ttl=250 time=0.528 ms

--- www.hs.fi ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 0.528/0.551/0.594 ms
root@router5:~# ping -c 4 www.yle.fi
PING www.yle.fi (52.85.49.45): 56 data bytes
64 bytes from 52.85.49.45: seq=0 ttl=248 time=0.624 ms
64 bytes from 52.85.49.45: seq=1 ttl=248 time=0.554 ms
64 bytes from 52.85.49.45: seq=2 ttl=248 time=0.560 ms
64 bytes from 52.85.49.45: seq=3 ttl=248 time=0.576 ms

--- www.yle.fi ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 0.554/0.578/0.624 ms
2 Likes

Crikey Charlie. I mean many places have FTTH without facilitating such low latencies. I wonder how come you see such low latencies. Finnish networking infrastructure is far more advanced?

Hello,

I am confused by the port selection.
You have to have 2 different named devices?
I currently have the following selection:
dl_if=wan # download interface
ul_if=br-wan # upload interface

Is this okay, correct?

You need to tell cake-autorate the interface(s) on which cake is applied so that it can send on the bandwidth adjustments to the appropriate qdisc(s).

Here is a link to: the INSTALLATION documentation (click me).

You can see which interfaces cake is applied on by issuing: tc qdisc ls.

Perhaps you could paste the output of that?

1 Like

Thanks.

Everything comes in and out of wan, also lan4.
Those are the two connections I am using.

root@OpenWrt:~# tc qdisc ls
qdisc noqueue 0: dev lo root refcnt 2
qdisc mq 0: dev eth0 root
qdisc fq_codel 0: dev eth0 parent :10 limit 10240p flows 1024 quantum 1518 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth0 parent :f limit 10240p flows 1024 quantum 1518 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth0 parent :e limit 10240p flows 1024 quantum 1518 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth0 parent :d limit 10240p flows 1024 quantum 1518 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth0 parent :c limit 10240p flows 1024 quantum 1518 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth0 parent :b limit 10240p flows 1024 quantum 1518 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth0 parent :a limit 10240p flows 1024 quantum 1518 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth0 parent :9 limit 10240p flows 1024 quantum 1518 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth0 parent :8 limit 10240p flows 1024 quantum 1518 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth0 parent :7 limit 10240p flows 1024 quantum 1518 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth0 parent :6 limit 10240p flows 1024 quantum 1518 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth0 parent :5 limit 10240p flows 1024 quantum 1518 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth0 parent :4 limit 10240p flows 1024 quantum 1518 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth0 parent :3 limit 10240p flows 1024 quantum 1518 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth0 parent :2 limit 10240p flows 1024 quantum 1518 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1518 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc mq 0: dev eth1 root
qdisc fq_codel 0: dev eth1 parent :10 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth1 parent :f limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth1 parent :e limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth1 parent :d limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth1 parent :c limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth1 parent :b limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth1 parent :a limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth1 parent :9 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth1 parent :8 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth1 parent :7 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth1 parent :6 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth1 parent :5 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth1 parent :4 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth1 parent :3 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth1 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc fq_codel 0: dev eth1 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc clsact ffff: dev eth1 parent ffff:fff1
qdisc cake 8015: dev wan root refcnt 2 bandwidth 10Mbit besteffort triple-isolate nonat nowash no-ack-filter split-gso rtt 100ms noatm overhead 18
qdisc clsact ffff: dev wan parent ffff:fff1
qdisc noqueue 0: dev lan1 root refcnt 2
qdisc clsact ffff: dev lan1 parent ffff:fff1
qdisc noqueue 0: dev lan2 root refcnt 2
qdisc clsact ffff: dev lan2 parent ffff:fff1
qdisc noqueue 0: dev lan3 root refcnt 2
qdisc clsact ffff: dev lan3 parent ffff:fff1
qdisc noqueue 0: dev lan4 root refcnt 2
qdisc clsact ffff: dev lan4 parent ffff:fff1
qdisc noqueue 0: dev sfp2 root refcnt 2
qdisc clsact ffff: dev sfp2 parent ffff:fff1
qdisc noqueue 0: dev br-lan root refcnt 2
qdisc clsact ffff: dev br-lan parent ffff:fff1
qdisc noqueue 0: dev br-wan root refcnt 2
qdisc clsact ffff: dev br-wan parent ffff:fff1
qdisc cake 8016: dev ifb4wan root refcnt 2 bandwidth 85Mbit besteffort triple-isolate nonat wash no-ack-filter split-gso rtt 100ms noatm overhead 18

Looks as if I need to select eth0, and eth1 ?
Those are the internal switches to br-wan, and br-lan.
I'd have to look up which is which.
The above output is not with the cake-autorate script running, just the luci-sqm, with piece of cake running.
BPI-R3 is the device, BTW...