CAKE w/ Adaptive Bandwidth [Historic]

autorate will relieve the pressure of getting the overhead compensation right somewhat, if the overhead is wrong (too small) the consequence is increased likelihood of encountering increased delays, but autorate is directly controlling via measureing the delay, so is more tolerant against getting the overhead wrong than a static sqm configuration. However, I still would aim at getting this right if just out of principle ;).
The wiki does not say anything about overhead on LTE links, because we have not found reliable information that allows us to figure that out precisely, but it seems that the applicable overhead is generally much smaller than 44 bytes so your configuration should be fine.

1 Like

Ah sorry, that test by default will report values that are 6%-age points higher than the actually measured values, which I consider sub-optimal (which I also informed the tests developer about).

The waveform test is not bad, but even Ookla's speedtest.net got much better recently (the on-line test now will show three latency values, one for idle, one for download and one for upload, not 100% clear whether that is the maximum RTT encountered during these phases or the median or mean, but it is better than the old idle only value; however these values are not yet reported/aggregated in the "results" overview page which collects multiple test results).

1 Like

Just did it but the script actually stopped logging for most of the test and showed this error:

./CAKE-autorate.sh: line 584: ((: (1658402778622798 - 091404: value too great for base (error token is "091404")

and now I can't start it again:

Error: /tmp/CAKE-autorate already exists. Is another instance running? Exiting script.

while I'm also not able anymore to "stop" the service (which worked before):

# service cake-autorate stop
Command failed: Not found

Looks like I messed something up :grimacing:

1 Like

Weird timestamp from ping response! Otherwise this is my bad - it's that leading 0 and so I need to specify base 10 to deal with this weird eventuality. I will look to see if it possible to set this globally because we've seen this issue come up elsewhere. I'll fix this in any case.

For now easy solution is either reboot router or kill all instances of bash/ping and rm /tmp/CAKE-autorate.

I did this

So newest test:
image
and the new paste for it :point_up:

Also wondering about the upper limits now. Just when I tested I spotted the download (without sqm/cake) to reach 2XMbps for a second and that reminded me actually that I do automatic speed tests on a regular interval :joy:

I think it's clearly visible when I activated the :cake: :point_up:

And yes, with longer history it's clear that our connections actually "performs" better (in terms of the max dl rate) than I remembered/expected:

1 Like

@MangoMan this looks pretty good to me.

Do you have visibility of your LTE stats? Curious that download capacity seems rather poorer than upload capacity. Is your cell tower heavily congested?

1 Like

Just watched the stats for some minutes and the range at the moment (very dry :hot_face: so best signal of the year) is something like that:

Signal Strength:	-71, -72, -74, -75, -76 dBm
RSRP:	-97 dBm
SINR:	0, 1, 2, 3, 5, 6, 7, 9 dB
RSRQ:	-7, -8, -9, -10, -11 dB

Yea, that's interesting but was always the case. My theory is that they put more "effort" in shaping the downstream (because it is what like 99% of the users mostly use) and not to much carrying about the upstream (maybe don't shape it at all)

Hey have you kept the autorate script running? If so could you provide the new graphs from your automatic speed tests / latency measurement?

How has latency / reactiveness been in general? Internet feel snappy?

Perhaps there is still room for improvement in the parameters.

Have you considered using an outdoor antenna? If not have you really tried optimizing position of the LTE modem? Mine works best face down on its side underneath a window between wall and floor. My neighbour's works best in an unused fireplace against side wall. There is typically a lot of room for improvement by adjusting the position of the modem. Tenacity pays off.

yes, indeed. With the last settings mentioned :point_up:

Here we go:

It is actually a outdoor antenna (with integrated modem).

It is already on the highest spot available installed (pver the roof)

Well, I don't need start inside the house as there is hardly any reception

1 Like

Like one can imagen with this tight budget available it's never a pleasure but I have the feeling the responsive of web surfing improved a bit (less waiting "ages" for a site to appear in the browser after sending a URL). On the other hand I'm not quite sure how about the situation with video streaming and if it tends to pauses more for buffering. Guess some more watching necessary here :joy:

New Idea - I'm not sure where to put it: I wrote up a challenge on Github to see if we could "misuse" CAKE-autorate to learn more about the underlying links. Here are my thoughts...

Enjoy!

I have:

Today my router went very laggy (wifi clients don't connect properly) and internal pings are huge by times and resolving host name is troublesome.

The luci interface isn't available but I get this "site" instead:

Bad Gateway
The process did not produce any response

Logging into ssh worked but took very long. Getting top to output the load average:

CPU:  61% usr  36% sys   0% nic   0% idle   0% io   0% irq   1% sirq
Load average: 25.63 18.79 15.93 8/313 28884

I wonder if cake-autorate is to blame? Should it run that often?

11537     1 root     R     2168   2%  17% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
16429     1 root     R     2140   2%  17% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
11527     1 root     R     2168   2%   7% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
11519     1 root     R     2168   2%   6% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
16462 16429 root     S     2096   2%   4% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
11525     1 root     R     2168   2%   2% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
11521     1 root     S     2168   2%   2% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
11517     1 root     S     2168   2%   2% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
11509     1 root     S     2168   2%   1% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
10583     1 root     S     2176   2%   1% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
11523     1 root     R     2168   2%   1% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
28827 28813 root     R     1288   1%   1% top
10951     1 root     S     2204   2%   1% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
10831     1 root     S     2196   2%   1% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
10513     1 root     S     2172   2%   1% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
10547     1 root     S     2172   2%   1% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
 1414     1 root     S     4376   4%   1% /usr/sbin/hostapd -s -g /var/run/hostapd/global
10963     1 root     S     2208   2%   1% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
10839     1 root     S     2196   2%   1% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
11184     1 root     S     2168   2%   1% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
10933     1 root     S     2204   2%   1% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
10881     1 root     S     2200   2%   1% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
10877     1 root     S     2200   2%   1% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
10791     1 root     S     2192   2%   1% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
10695     1 root     S     2184   2%   1% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
11223     1 root     S     2168   2%   1% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
    7     2 root     RW       0   0%   1% [ksoftirqd/0]
10925     1 root     S     2204   2%   0% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
10809     1 root     S     2196   2%   0% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
10737     1 root     S     2188   2%   0% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
10733     1 root     S     2188   2%   0% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
10681     1 root     S     2184   2%   0% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
10641     1 root     S     2180   2%   0% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
10561     1 root     S     2176   2%   0% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
10563     1 root     R     2176   2%   0% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
11530     1 root     R     2168   2%   0% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
11246     1 root     S     2168   2%   0% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
11231     1 root     S     2168   2%   0% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh

I did run `services cake-autorate stop" now (I checked an the "services" was running).
It took like 3 minutes or more to finish and the web interface somewhat came available again. Still ssh and luci are the opposite of responsive.

I guess I need a hard restart now (probably then all logs will be lost to debug this :frowning: )

Hmm. Is there anything in the log file in /tmp? From top looks like you had many instances running simultaneously unless you have increased the number of pingers to well beyond the default (4).

The cpu stats are misleading if you have schedutil enabled since it clocks down the processor to the minimum needed to keep everything running.

I spent quite a lot of time tweaking the bash implementation to get the CPU down as far as possible, but bash is still bash. For my sub 100Mbit/s connection the CPU use on my RT3200 is sufficiently minor not to affect anything with default settings. the settings could be tweaked to reduce CPU usage of desired.

I didn't restart yet but it's impossible to reach it :frowning: ...so I power cycled the device

I kept the defaults for everything (only set the dl/ul min/max..)

Does CAKE eat my RAM? Linux ate it!

My luci web interface is everything from responsive and while it now looks good again memory wise:

one minute ago I only had like 1 MiB available :grimacing:

Establishing a ssh connection takes a good a amount of seconds before I even asked for the password and even more seconds afterwards. Then I get only half of the greeting:



BusyBox v1.33.2 (2022-02-16 20:29:10 UTC) built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt 21.02.2, r16495-bf0c965af0
 -----------------------------------------------------

and like a good minute later just the command prompt which is everything apart from responisve :thinking:

I just enabled fast transition on wifi - maybe this hogs on memory together with :cake:?

Total :pinching_hand: available now:

Who ate my ram?

My cake-autorate options regarding output/debug are all :zero:

# *** OUTPUT OPTIONS ***

output_processing_stats=0 # enable (1) or disable (0) output monitoring lines showing processing stats
output_cake_changes=0     # enable (1) or disable (0) output monitoring lines showing cake bandwidth changes
debug=0			  # enable (1) or disable (0) out of debug lines

EDIT: just checking top looks like upnp is going crazy (CPU wise) actually:

15131  1086 root     R     1176   1%  79% lua /usr/libexec/rpcd/luci.upnp call get_status

but for the cake-autorate script - is it normal it runs many times in parallel?

15970     1 root     S     2208   2%   0% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
15954     1 root     S     2208   2%   0% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
15952     1 root     S     2208   2%   0% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
15948     1 root     S     2208   2%   0% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
15950     1 root     S     2208   2%   0% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
15958     1 root     S     2208   2%   0% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
15968     1 root     S     2208   2%   0% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
15960     1 root     S     2208   2%   0% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
15963     1 root     S     2208   2%   0% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
15965     1 root     S     2208   2%   0% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
15956     1 root     S     2208   2%   0% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
15920     1 root     S     2204   2%   0% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
15930     1 root     S     2204   2%   0% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
15918     1 root     S     2204   2%   0% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
15922     1 root     S     2204   2%   0% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
15924     1 root     S     2204   2%   0% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
15926     1 root     S     2204   2%   0% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh
15911     1 root     S     2204   2%   0% {CAKE-autorate.s} /bin/bash /root/CAKE-autorate/CAKE-autorate.sh

EDIT:

Restarting the router did it bring back onto a normal level:

I also have less CAKE-autorate scripts running now (like 7 I see in top). :thinking:

Each cake instance by default (in OpenWrt) will at most take 4 MB for its queues per instance, so a normal install with one upload and one download instance should not take much more than 2 * 4 = 8 MB.

1* script instances sounds like a lot, but how many latency reflectors do you have configured?

Mmmh, do you have cases where the script has to replace reflectors? @Lynx in case of a stuck reflector do we kill the old process that pinged that reflector for good?

Yes we kill off bad ping process here:

And with sleep will kill off all ping processes here:

@MangoMan are you using a service file? Although the script shouldn't allow multiple instances to run by exiting where its tmp folder already exists.

Can you also try setting sleep duration really short and threshold high to make it sleep often and verify that the processes are not increasing?

Or try setting bad reflectors so we get continual rotation and also monitor processes?

We should work out why you are seeing duplicate processes. Really hope you can help me work out what's going wrong.

But it's not affecting me at all.

@moeller0 can you see anything wrong with the kill processes?

On my system when the pipe is broken because the fifo file is removed the corresponding monitor background process is also killed. Could it be on some systems that doesn't happen for some reason?

Pipe is here:

Yes, went exactly like described in the githubs.

Any hint were to find that config? Probably in the conf file I expect?

It's weird - but it might be the second time now that it happens (router got already unresponsive in the past ones since I deployed :cake:)... but I guess it takes many days that it hogs on all the ram available..

@MangoMan you have disabled output logging when running as service right? Else the log file will grow very quickly and eat up router ram.

1 Like

this is my config at this very moment:

anything more I missed eventually?