CAKE w/ Adaptive Bandwidth [August 2022 to March 2024]

Just to be clear, when you look at CPU usage, you’re summing the usage of your core bash script plus all the external processes it calls, yeah?

Yes. And stay calm; nobody is going to try to argue that bash is faster than Lua. But I'm definitely going to argue that there is no magic bullet.

Pretty sure the real reason we abandoned work on Lua version was that the one line patch couldn't be pushed through. without it you can't run the script without a version of Lua built with the patch which everyone on the dev team was running no problem, but we weren't going to build and distribute separate Lua packages for every architecture for OpenWrt.

1 Like

Oh I’m calm over here. I actually was laughing when I saw you dig up an old issue about CPU usage on a router from 2013. I had a hard time believing that even was brought into discussion here.

Anyway, do as you will with bash. The Lua code is out there and anyone who wants to tinker with it can. I’m not going to invest time into the Python route right now.

Got bigger things demanding my time :slightly_smiling_face:

1 Like

I don't believe we even need a patch. All that's actually required is updating the LuaLanes package to 3.16.0.

See upstream patch here: https://github.com/LuaLanes/lanes/commit/c8466a0c51422c6a63009a7220137ca8c4ae1a79

Edit: I forgot I wrote instructions to compile an updated LuaLanes package in the GitHub issue

1 Like

Fair enough, as I said this is more about "scratching one's own itches" and if these itches do not overlap fully the "scratching moves" will differ. Personally I was considering starting from scratch to employ what we learned in a clean fashion, but then I felt this would end up deeply in "second system syndrome" (trying to do everything much better often not resulting in a viable solution but something dragged down by its ambitions) and decided to see whether I can evolve from an more or less arbitrarily selected starting point* into something that shows how different aspects of the problem can be tackled. This still might end up in failure, but that is fine even in that case I will learn something.
As I said, I am fine with sticking to bash, as it is IMHO still a decent fit for the requirements (well it fails the compiler&interpreter "requirement" but it is not alone in that). I will however need to simplify a few things (but just because my bash experience is limited and I want to understand the code).

*) Not fully arbitrary, but from a point where we wanted to cut a 1.2.0 release and before the recent big rework of many aspects as I am slow and do not like too much relevant changes happening under my feet while I try to solve something particular.

Is there a way to use this on Asuswrt Merlin?

Is there any appetite for considering zsh instead of bash? I imagine the size might be of concern to some, but if zsh were a contender, it supports some more advanced constructs that could prove helpful.

For example:
https://info2html.sourceforge.net/cgi-bin/info2html-demo/info2html?(zsh)The%2520zsh%2Fnet%2Fsocket%2520Module

Here is an example worker queue (adapted from here) based on a zsocket and tested on my RT3200:

Example ZSocket Worker Queue
#!/usr/bin/env zsh

zmodload zsh/net/socket


producer() {
  typeset -i threads="$1"
  typeset -i deadcount=0
  shift

  typeset -a myarg=("$@")
  zsocket -l /tmp/testsock
  typeset sock=$REPLY

  (

    while zsocket -a $sock
    do
      typeset fd=$REPLY

      typeset narg=${myarg[1]}
      if [[ -n $narg ]]
      then
        print "$narg" >&$fd
        shift myarg
        exec {fd}>&-
      else
        print '\000' >&$fd
        (( deadcount++ ))
        exec {fd}>&-
        if (( deadcount >= threads ))
        then
          rm -v /tmp/testsock
          break
        fi
      fi
    done


  ) &
}

worker() {
  worker_num=$1
  while true
  do
    zsocket /tmp/testsock
    typeset -i fd=$REPLY
    myarg="$(<&$fd)"
    exec {fd}>&-
    if [[ $myarg == $'\000' ]]
    then
      print "Worker $worker_num ended."
      break
    elif [[ -z $myarg ]]
    then
      #CAN'T HAPPEN
      print "Worker $worker_num takes EMPTY ARGUMENT!!!"
      exit 127
    else
      print -l "Worker $worker_num: $myarg"
    fi
  done
}

producer 3 {1..100}
( worker 1 ) &
( worker 2 ) &
( worker 3 ) &
wait

I think @anon58727419 was showing some interest in zsh, so there might be some interest.

But are modules part of OpenWrt's zsh package?

Personally, no, as I keep repeating, I do not believe that my current issues with the code are caused by limits from bash (but my limits with logic :wink: ). I first want to see whether I can pummel my toy version into a shape and form I am happy with and get it to operate robustly and reliably at the same time. After I reached that state (or accept that I failed), I am happy to re-consider implementation choices, but I really first want something that does what it is supposed to do.

Yes, a port was written - see here:

But ideally that would get updated to the code now in the 'master' branch since it is based on an older version. You could try posting here:

hi everyone. I am giving a talk on tuesday and would like to use a graph that also points to the amazing work done here. 1) I don't know which one to use, there has been so many! a before/after view would be nice, and something that speaks to a more general audience. Suggestions? 2) I hereby request permission to use one of your favorites in my talk.

Free signup here, you might recognize a few of the speakers. :slight_smile: Notably kathie nichols is keynoting tuesday!

1 Like

Hi Dave,

any of my tests/plots are "fair game" and I am happy to share, however my link does not really profit from autorate so is not a good example of what the whole exercise is for (normally I get a 116/37 Mbps sync which results in goodput in the 105/33 range) even without cake my ISP bounds the worst case bufferbloat to <= 150 ms (don't get me wrong 150ms is still atrocious, but far from the multiple seconds we saw in the past that make easy to spot "before" and "after" plots)*
While I am short on time, I am happy to help modifying the plotting code if you have special requirements/ideas.

) Recently my ISP started routing me through either an overloaded PPPoE terminator or some broken gear, in one IPv4 "prefix" I see 1-2% packet loss independent of throughput (so with a 500 sample mtr at night to different typically well connected DNS servers). That is pretty grotty, but exists even without any load, so autorate does not help. (@Lynx it is when I am in that prefix that autorate will go bonkers quickly, I think this is related to high loss tickling the old stall detection too often*)

**) Probably a non issue in 2.0, but my version still has issues in that regard.

is there any other code that works with irtt? My attempt to use the julia irtt code failed, so did the python, I am hopeless...

thank you for your permission for your images. It was, unfortunately, one or more of the more extreme cases posted here that I think I would be most interested in, and all I have been doing is admiring y'alls work from afar.

2 Likes

I bailed on that as it requires json parsing, and the json parser for matlab/octave I tried first took ages and the still failed... I guess for the way forward would likely be to use python ot export the json inot something easier to read from octave... Yes I know that python sports matplotlib, a library inspired by matlabs convenient plotting functions, but that still does not help my use that python code quickly.

That what I thought, however al I can offer is my own plots, I would hope that @Lynx has pretty before and after plots, and maybe one of the starlinkers like @gba? My offer to help with formatting re-plotting still stands though....

P.S.: Personally I feel posting plots in a public forum essentially implicitly contains permission for respectful re-use, but I am also aware that my feeling is not aligned with "the law" (assuming there is a single law applicable and not multiple similar laws in multiple countries)

I dropped you a quick note via email. I am in the too frustrated to have something fit into 15 slides for this audience stage... bbl

1 Like

@dtaht , Dave if you can get yourself an irtt dataset you like, I will happily make the Julia plotter output a nice plot.

For those who want to try it.

@dtaht feel free to use any of the plots or data I've posted on this or on the historical thread.

1 Like

I've only ever used my perl implementation - as you know I've been running and developing it since 2020. I and haven't made any major changes for months, and in its current form it runs without a hitch on my router for weeks on end.

I strongly suspect that I'm the only person using it, but that honestly doesn't bother me at all. I do try to follow this thread and I'm always happy to discuss stuff and contribute where possible, but my ambitions have never really extended beyond solving latency problems in my own home.

1 Like

You've been ahead of the game for so long and it's been a case of catch-up. Your insights along the way have been immensely helpful.

I am in a similar boat in that I just keep chipping away partly because I really need this tool for my own 4G connection, although I have tried my best to put together something that might be useful for others too. I have it on 24/7 and on my setup it just works, albeit other users have encountered issues and I've worked hard to try to address those. I am feeling pretty good about the way things have been progressing recently, thanks in part to input from @moeller0, @anon58727419 and @patrakov.

You may well have some users already that you don't know about. Users tend to point out problems far more readily than success stories.

I'd encourage readers of this thread to try out the perl version linked at the top of this thread if they have not already.

2 Likes

thank you! I did download julia, did install all the dependencies, and the script runs and exits (on my chromebook) without doing anything.

So here is a back to back run with irtt + speedtest.net ( https://www.speedtest.net/result/14438617607 )

and irtt + cloudflare's new speedtest. (they don't have a number, let me paste a screenshot)

These are the irtt files obtained with a 10ms interval running for one minute each.

http://www.taht.net/~d/irttwithspeedtest.json.gz

http://www.taht.net/~d/irttwithcloudflarespeedtest.json.gz

I am pretty sure both will be headscratchers. Loss? delay? No idea. Not running sqm-autorate...

1 Like