CAKE w/ Adaptive Bandwidth [August 2022 to March 2024]

Aha. at the shell command line, cd to the directory where the code is, make sure your irtt output is in a file named testdata.json and do:

julia --project

you will wind up at a julia REPL prompt. Type

include("plotirtt.jl")

you should wind up with a window with the plot it in. To save that to a file you can do

savefig("filename.png")   #(or use .pdf or .svg)

you should get the most recent figure saved to the named file. You will still be at the julia command line. You can call plotirttfile("some_other_filename.json") to get another plot. then savefig("some_second_file_name.png") to save that... etc

I'm happy to enhance the plots in various ways (colors, line thickness, labels, multiple timeseries, whatever). I'd be happy to run the plots, but if you can give the above a try first, that might be faster and better for you to wind up getting what you want. Ctrl-D at the julia prompt will make it quit.

Note these plots were VERY simplistic, just a first quick pass. Like I said, happy to revise them to something that looks like what you want if you can give suggestion as to what that is.

I am not sure if you want to teach an old fart this much julia, but...

 Loading Pkg into OpenLibm_jll from project dependency, future warnings for OpenLibm_jll are suppressed.
ERROR: LoadError: ArgumentError: Package CompilerSupportLibraries_jll does not have JLLWrappers in its dependencies:
- If you have CompilerSupportLibraries_jll checked out for development and have
  added JLLWrappers as a dependency but haven't updated your primary
  environment's manifest file, try `Pkg.resolve()`.
- Otherwise you may need to report an issue with CompilerSupportLibraries_jll
Stacktrace:
 [1] require(::Module, ::Symbol) at ./loading.jl:906
 [2] include(::Function, ::Module, ::String) at ./Base.jl:380
 [3] include(::Module, ::String) at ./Base.jl:368
 [4] top-level scope at none:2
 [5] eval at ./boot.jl:331 [inlined]
 [6] eval(::Expr) at ./client.jl:467
 [7] top-level scope at ./none:3
in expression starting at /home/dtaht/.julia/packages/CompilerSupportLibraries_jll/790hI/src/CompilerSupportLibraries_jll.jl:5
ERROR: LoadError: LoadError: Failed to precompile CompilerSupportLibraries_jll [e66e0078-7015-5450-92f7-15fbd957f2ae] to /home/dtaht/.julia/compiled/v1.5/CompilerSupportLibraries_jll/iCwSB_7u3Ll.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1305
 [3] _require(::Base.PkgId) at ./loading.jl:1030
 [4] require(::Base.PkgId) at ./loading.jl:928
 [5] require(::Module, ::Symbol) at ./loading.jl:923
 [6] include(::Function, ::Module, ::String) at ./Base.jl:380
 [7] include(::Module, ::String) at ./Base.jl:368
 [8] top-level scope at /home/dtaht/.julia/packages/JLLWrappers/QpMQW/src/toplevel_generators.jl:188
 [9] include(::Function, ::Module, ::String) at ./Base.jl:380
 [10] include(::Module, ::String) at ./Base.jl:368
 [11] top-level scope at none:2
 [12] eval at ./boot.jl:331 [inlined]
 [13] eval(::Expr) at ./client.jl:467
 [14] top-level scope at ./none:3
in expression starting at /home/dtaht/.julia/packages/OpenSpecFun_jll/Xw8XK/src/wrappers/x86_64-linux-gnu-libgfortran5.jl:4
in expression starting at /home/dtaht/.julia/packages/OpenSpecFun_jll/Xw8XK/src/OpenSpecFun_jll.jl:8
ERROR: LoadError: Failed to precompile OpenSpecFun_jll [efe28fd5-8261-553b-a9e1-b2916fc3738e] to /home/dtaht/.julia/compiled/v1.5/OpenSpecFun_jll/TDl1L_7u3Ll.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1305
 [3] _require(::Base.PkgId) at ./loading.jl:1030
 [4] require(::Base.PkgId) at ./loading.jl:928
 [5] require(::Module, ::Symbol) at ./loading.jl:923
 [6] include(::Function, ::Module, ::String) at ./Base.jl:380
 [7] include(::Module, ::String) at ./Base.jl:368
 [8] top-level scope at none:2
 [9] eval at ./boot.jl:331 [inlined]
 [10] eval(::Expr) at ./client.jl:467
 [11] top-level scope at ./none:3
in expression starting at /home/dtaht/.julia/packages/SpecialFunctions/gXPNz/src/SpecialFunctions.jl:19
ERROR: LoadError: Failed to precompile SpecialFunctions [276daf66-3868-5448-9aa4-cd146d93841b] to /home/dtaht/.julia/compiled/v1.5/SpecialFunctions/78gOt_7u3Ll.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1305
 [3] _require(::Base.PkgId) at ./loading.jl:1030
 [4] require(::Base.PkgId) at ./loading.jl:928
 [5] require(::Module, ::Symbol) at ./loading.jl:923
 [6] include(::Function, ::Module, ::String) at ./Base.jl:380
 [7] include(::Module, ::String) at ./Base.jl:368
 [8] top-level scope at none:2
 [9] eval at ./boot.jl:331 [inlined]
 [10] eval(::Expr) at ./client.jl:467
 [11] top-level scope at ./none:3
in expression starting at /home/dtaht/.julia/packages/ColorVectorSpace/QI5vM/src/ColorVectorSpace.jl:3
ERROR: LoadError: Failed to precompile ColorVectorSpace [c3611d14-8923-5661-9e6a-0046d554d3a4] to /home/dtaht/.julia/compiled/v1.5/ColorVectorSpace/7uC4N_7u3Ll.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1305
 [3] _require(::Base.PkgId) at ./loading.jl:1030
 [4] require(::Base.PkgId) at ./loading.jl:928
 [5] require(::Module, ::Symbol) at ./loading.jl:923
 [6] include(::Function, ::Module, ::String) at ./Base.jl:380
 [7] include(::Module, ::String) at ./Base.jl:368
 [8] top-level scope at none:2
 [9] eval at ./boot.jl:331 [inlined]
 [10] eval(::Expr) at ./client.jl:467
 [11] top-level scope at ./none:3
in expression starting at /home/dtaht/.julia/packages/ColorSchemes/NiGkg/src/ColorSchemes.jl:16
ERROR: LoadError: Failed to precompile ColorSchemes [35d6a980-a343-548e-a6ea-1d62b119f2f4] to /home/dtaht/.julia/compiled/v1.5/ColorSchemes/EO5fj_7u3Ll.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1305
 [3] _require(::Base.PkgId) at ./loading.jl:1030
 [4] require(::Base.PkgId) at ./loading.jl:928
 [5] require(::Module, ::Symbol) at ./loading.jl:923
 [6] include(::Function, ::Module, ::String) at ./Base.jl:380
 [7] include(::Module, ::String) at ./Base.jl:368
 [8] top-level scope at none:2
 [9] eval at ./boot.jl:331 [inlined]
 [10] eval(::Expr) at ./client.jl:467
 [11] top-level scope at ./none:3
in expression starting at /home/dtaht/.julia/packages/PlotUtils/VgXdq/src/PlotUtils.jl:3
ERROR: LoadError: Failed to precompile PlotUtils [995b91a9-d308-5afd-9ec6-746e21dbc043] to /home/dtaht/.julia/compiled/v1.5/PlotUtils/YveHG_7u3Ll.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1305
 [3] _require(::Base.PkgId) at ./loading.jl:1030
 [4] require(::Base.PkgId) at ./loading.jl:928
 [5] require(::Module, ::Symbol) at ./loading.jl:923
 [6] include(::Function, ::Module, ::String) at ./Base.jl:380
 [7] include(::Module, ::String) at ./Base.jl:368
 [8] top-level scope at none:2
 [9] eval at ./boot.jl:331 [inlined]
 [10] eval(::Expr) at ./client.jl:467
 [11] top-level scope at ./none:3
in expression starting at /home/dtaht/.julia/packages/RecipesPipeline/bexlW/src/RecipesPipeline.jl:6
ERROR: LoadError: Failed to precompile RecipesPipeline [01d81517-befc-4cb6-b9ec-a95719d0359c] to /home/dtaht/.julia/compiled/v1.5/RecipesPipeline/QQIne_7u3Ll.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1305
 [3] _require(::Base.PkgId) at ./loading.jl:1030
 [4] require(::Base.PkgId) at ./loading.jl:928
 [5] require(::Module, ::Symbol) at ./loading.jl:923
 [6] include(::Function, ::Module, ::String) at ./Base.jl:380
 [7] include(::Module, ::String) at ./Base.jl:368
 [8] top-level scope at none:2
 [9] eval at ./boot.jl:331 [inlined]
 [10] eval(::Expr) at ./client.jl:467
 [11] top-level scope at ./none:3
in expression starting at /home/dtaht/.julia/packages/StatsPlots/CpWiU/src/StatsPlots.jl:7
ERROR: LoadError: Failed to precompile StatsPlots [f3b207a7-027a-5e70-b257-86293d7955fd] to /home/dtaht/.julia/compiled/v1.5/StatsPlots/SiylL_7u3Ll.ji.
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] compilecache(::Base.PkgId, ::String) at ./loading.jl:1305
 [3] _require(::Base.PkgId) at ./loading.jl:1030
 [4] require(::Base.PkgId) at ./loading.jl:928
 [5] require(::Module, ::Symbol) at ./loading.jl:923
 [6] include(::String) at ./client.jl:457
 [7] top-level scope at REPL[1]:1
in expression starting at /home/dtaht/git/IrttPlot/plotirtt.jl:1
1 Like

julia> Pkg.resolve()
ERROR: UndefVarError: Pkg not defined

whoa. hmmm... did not expect that.

did you install julia from the julialang site or some kind of older package from a distro like Debian or Red Hat or whatever? (looking at the output, you seem to be running 1.5 which is ... pretty old)

I strongly recommend just downloading this on an x86 linux machine and untarring it, then run that version:

https://julialang-s3.julialang.org/bin/linux/x64/1.8/julia-1.8.5-linux-x86_64.tar.gz

Or if you're somehow on windows...

https://julialang-s3.julialang.org/bin/winnt/x64/1.8/julia-1.8.5-win64.exe

I was foolish enough to get a shiny new chromebook for christmas. Love the battery life, not much else. Related rant here: https://blog.cerowrt.org/post/carping_on_a_chromebook/

I will try installing it that way. Thx!

1 Like

I have tested the latest master (32b0bac129b3f265558b3580b3f01db0f266afcd), on LTE (the ISP still hasn't fixed the GPON link) and I have to say that it is (as well as static SQM), with my settings, worse than nothing on my connection for real business usage (Discord calls). Consumer-style activities (YouTube, web browsing, even light games) are OK. Reminder: this is with marginal LTE signal (CSQ=16, RSRP=-114 dBm, SINR=-5 dB, RSRQ=-12 dB).

Please discard all my other "OK" reports - they were just tests for it not crashing, and do not represent real business use. Perhaps something to tune again regarding bufferbloat_detection_thr and the high_load_thr (0.80 is correct above, let's say, 3-4 Mbps, but below that, it is wrong).

Basically, without SQM at all, when link conditions are not the worst ones, and there is no other traffic, Discord video calls work. With cake-autorate, it decreases the bandwidth to the minimum, because it sometimes detects bufferbloat. The bloat would then resolve by itself in a second, but it's too late, as the bandwidth increase doesn't happen. OTOH, during evenings, when the signal is even worse, SQM is necessary for web pages to load at all. Perhaps the conclusion is, there is no such thing as "static rate below which I prefer bandwidth to latency" (our min_dl_shaper_rate_kbps).

Config: https://pastebin.com/2PUjzCQZ

Log: https://pastebin.com/Wr4aYxP2

EDIT: and right after the meeting, the GPON line has been fixed.

BTW that's 1.2.0 right? Sub 5Mbit/s capacity, spurious latency spikes, and those poor LTE stats represents a really significant challenge.

No.

commit 32b0bac129b3f265558b3580b3f01db0f266afcd (HEAD -> master, origin/master)
Merge: c79bb87 e69cdbc
Author: lynxthecat <***>
Date:   Sat Mar 4 19:34:03 2023 +0000

    Merge pull request #170 from rany2/master
    
    Add missing brace on creation of export script

Look at your config and log. They both report 1.2.0.

That's because I did not update the config (which is a separate file, not in git) when upgrading past 1.2.0.

Then set min_dl_shaper_rate_kbps to zero... and the controller will scale down all the way... But my take is that you want something else instead and that is a less sensitive controller that will soldier on over transient bumps?

2.0.0 uses a different config format and the log will show 2.0.0 running.

It could be that your settings aren't reflecting what you want or need. Or that you need a better LTE connection. Or there is more we could do to work with such a connection. I'm open minded.

I thought that in 2.0.0 the majority of the config was moved to cake-autorate_defaults.sh. I thought that just copying the old config would override everything, and thus be still valid. Is it not true?

Regarding the need for a "better LTE connection" - well, as Discord works well during daytime without cake-autorate, this is ruled out, I think. The other two operators are even worse.

1 Like

So as explained above and in the readme there is now the set of defaults and interface names, whether to apply shaper rates, bandwidths and overrides are now placed in the config. But again your log shows running 1.2.0.

As I said I am open minded, but my point is that your connection presents a significant challenge. Spurious latency spikes are mixed in with real bufferbloat. How much have you tried playing with the settings?

That's because the config says, wrongly,

cake_autorate_version="1.2.0"

I still maintain it is valid to override everything by copying the old config.

Oh I see. Well yes maybe. Although that's really not what was intended.

Maybe you could run your data through @moeller0's Octave plotter? Would be nice to see what's going on. I can't easily plot myself right now.

Well, I think it would be a good idea. There is another optional internal meeting in a few hours, I will try to attend it from LTE and to record half of it with cake-autorate on and half of it with adjust_{dl,ul}_shaper_rate=0.

Maybe this should not be set in a user-configurable file in the first place? :wink: it is not that one could change versions by editing this variable.

1 Like

Please take to separate log files for the two conditions, otherwise the CDF plots will simply mix up both conditions. Or if you want a unified log, use the option to only plot certain time ranges and do this separately for each of the two conditions.

1 Like