by the way the only stuff that i've change from default are:
rtt 100ms to 30ms
triple-isolate to dual-srchost / dual-dsthost
nonat to nat
and only the custom speed limits. Everything else is default.
My initial idea was to trying figure out the best to reduce ingame latecy with 4g, i'm not changing anything else xD, my brain is nothing compared with yours. I think i already achieve good results. I'll be more than welcome to be testing and documeting this subject with GUIDANCE (in terms of testing). Thank you for everything, and i'm sorry for any time wasted. Kind regards
I think the defaults rather than my overrides would likely be better for online gaming. My overrides give quite a relaxed controller that tolerates a fair degree of bufferbloat. Whereas for online gaming I think you really want to avoid bufferbloat.
Always happy to help and not at all. No time wasted. I can help you get some data from speed tests. That’d be good to see how things are performing.
Have you managed to obtain output SUMMARY lines from the console?
Got a summer house in the middle of nowhere connected over two crappy, oversold and overloaded 4G links, so kind of evaluating my options on reigning in latency spikes.
A few comments on cake-dual-ifb.nft:
"define IFACE_IPS" :
nft sets can't hold both IPv4 and IPv6, set definition and matching rule should be split.
Could be royal PITA to keep local IPv6 prefix filters updated for people with dynamic GUA though.
At the moment local intra-network IPv6 flows are needlessly marked.
table inet hook output
Output chain only sets fwmark=2 for router->wan flows, it should also mark router->tunnel-over-wan flows. Although I have no idea if flows from GRE, SIT, IPIP or Wireguard tunnels can be reliably mirrored to cake interfaces.
mark !=3 hack can only work for Wireguard tunnel configured with such fwmark.
Flows to GRE/SIT/IPIP and similar tunnel endpoints over then WAN interface have to be filtered based on destination address + protocol, or destination address + port for UDP based tunnels.
In general, output and forward hooks aren't the best places to mess with nft marks, since marks are typically used for policy based routing, and reroute check happens after output/forward hooks. Perhaps marking logic can be consolidated and moved to postrouting hook.
dscp_set_*
dscp chains only process IPv4, IPv6 should be added.
But what makes me puzzled the most is the way lan->wan flows are filtered in bridge table. Guess usage of bridge table is somehow related to tc mirroring, got to mark packets early? But I got two physical wan links, and there is no theoretical way to sort out flows before routing (and rerouting) happens. Implicit assumption "ip daddr != $IFACE_IPS" just doesn't work here.
This thread is primarily for cake-autorate rather than setting up cake correctly in the first place. IFBs can be built up with whatever flows you want in order to apply cake to the right mixture.
I didn't even get to cake or cake-autorate itself yet. INSTALLATION.md recommends cake-dual-ifb for PBR scenarios, which in turn has several issues I've described in my post.
cake-autorate will not instantiate cake for you, so you need a method to start cake for the desired directions, like cake-qos-simple or sqm-scripts, then you essentially need to get the two interfaces from the running cake instances. If you have done this tc -s qdisc should give some help to figure out the answer to your question.
Conceptually easy (the devil is in the details though): maximal rate: You either do not want or can not exceed that rate, e.g. on a 1 Gbps ethernet link 1000000 Kbps is a hard maximum, no need having autorate trying larger values. Or you might know that on your router CPU cake will top out at say X Kbps, again not much use in allowng autorate to probe higher settings. base rate: This is the rate you expect to work most of the time (e.g. run a few capacity tests over the course of the day and pick a number you can reliqably achieve) minimal rate: This is the rate below which you do not value low latency above capacity, so even if the delay is large/increasing autorate will never set the shaper below this rate
Thank you for your advice. Running tc -s qdisc helped me identify the download and upload interfaces, which are "wan" and "ifb4wan" (I had only tried "ifb4-wan" earlier, as mentioned in the GitHub documentation, but it didn't work).
I believe the script is now running, but when I check its status with ps | grep cake-autorate, I see the following results:
5182 root 1896 S bash /root/cake-autorate/launcher.sh
5184 root 2556 S bash /root/cake-autorate/cake-autorate.sh /root/cake-autorate/config.primary.sh
5294 root 2256 S bash /root/cake-autorate/cake-autorate.sh /root/cake-autorate/config.primary.sh
5298 root 2200 S bash /root/cake-autorate/cake-autorate.sh /root/cake-autorate/config.primary.sh
5310 root 2328 S bash /root/cake-autorate/cake-autorate.sh /root/cake-autorate/config.primary.sh
5634 root 1100 R grep cake-autorate
Is it normal to have so many instances running? Even after I stop the script and restart it, the same results remain.
I would appreciate it if you could let me know if this is an error and if I should re-do the entire process. I'm a beginner with this, and I really appreciate your help.
I have to defer to @lynx, as I have not been running any recent version of cake-autorate and there where lots of clean-up/speed-up changes since I last looked. My guess is that there might be 4 processes running all doing different jobs, but since these are instances called from within the main script they share the same command line, but then, as I said, I might simply be out of date.
Also these might be left overs from your experimentation, when you properly stop cake-autorate all of these should disappear, if they do things look benign, if however some of these survive that stopping this looks more like something to fix.
Sidenote: This was one reason for me to contemplate to have each process live in its own uniquely names script so the ps output would be easier to understand.
Maybe consider moving the respective functions into properly named files (assuming we are talking about 4 different functions here) so that the ps output becomes more useful (this might also help with debugging with strace a bit )
Hi!
Today I upgraded the router Linux box from Buster to Bullseye, where cake-autorate is running.
While checking if things are ok, I noticed a high cpu consumption from cake-autorate.
Is is expected?
Is this with frequency scaling putting the clock rate down to the minimum considered necessary (which will distort CPU usage) or with the performance governor?
What's the ping frequency? CPU use is directly proportional to that.
Is this with all the logging options enabled? CPU use goes up with logging lines, especially at high ping response frequency. You can help reduce CPU use by increasing the log file buffer size:
With the default 20Hz reflector response and only SUMMARY lines output to the log, I see a total CPU consumption of circa 9% on my RT3200 router.
gustavo@srv:/var/log$ sudo cpupower frequency-info
[sudo] password for gustavo:
analyzing CPU 0:
driver: intel_cpufreq
CPUs which run at the same hardware frequency: 0
CPUs which need to have their frequency coordinated by software: 0
maximum transition latency: 20.0 us
hardware limits: 1.33 GHz - 2.42 GHz
available cpufreq governors: performance schedutil
current policy: frequency should be within 1.33 GHz and 2.42 GHz.
The governor "performance" may decide which speed to use
within this range.
current CPU frequency: Unable to call hardware
current CPU frequency: 2.42 GHz (asserted by call to kernel)
boost state support:
Supported: yes
Active: yes
These are the only changes I do to the defaults:
#!/bin/bash
# *** STANDARD CONFIGURATION OPTIONS ***
log_to_file=1 # enable (1) or disable (0) output logging to file (/tmp/cake-autorate.log)
log_file_max_time_mins=20 # maximum time between log file rotations
log_file_max_size_KB=20000 # maximum KB (i.e. bytes/1024) worth of log lines between log file rotations
### For multihomed setups, it is the responsibility of the user to ensure that the probes
### sent by this instance of cake-autorate actually travel through these interfaces.
### See ping_extra_args and ping_prefix_string
dl_if=ifb4enp2s0 # download interface
#dl_if=enp2s0 # download interface
ul_if=enp2s0 # upload interface
# Set either of the below to 0 to adjust one direction only
# or alternatively set both to 0 to simply use cake-autorate to monitor a connection
adjust_dl_shaper_rate=1 # enable (1) or disable (0) actually changing the dl shaper rate
adjust_ul_shaper_rate=1 # enable (1) or disable (0) actually changing the ul shaper rate
min_dl_shaper_rate_kbps=200000 # minimum bandwidth for download (Kbit/s)
base_dl_shaper_rate_kbps=325000 # steady state bandwidth for download (Kbit/s)
max_dl_shaper_rate_kbps=450000 # maximum bandwidth for download (Kbit/s)
#min_ul_shaper_rate_kbps=10000 # minimum bandwidth for upload (Kbit/s)
#base_ul_shaper_rate_kbps=22000 # steady state bandwidth for upload (KBit/s)
#max_ul_shaper_rate_kbps=35000 # maximum bandwidth for upload (Kbit/s)
min_ul_shaper_rate_kbps=10000 # minimum bandwidth for upload (Kbit/s)
base_ul_shaper_rate_kbps=35000 # steady state bandwidth for upload (KBit/s)
max_ul_shaper_rate_kbps=85000 # maximum bandwidth for upload (Kbit/s)
#dl_delay_thr_ms=20.0 # (milliseconds)
#ul_delay_thr_ms=20.0 # (milliseconds)
reflectors=(
<redacted >)
#achieved_rate_adjust_down_bufferbloat=0.85 # how rapidly to reduce achieved rate upon detection of bufferbloat
#shaper_rate_adjust_down_bufferbloat=0.85 # how rapidly to reduce shaper rate upon detection of bufferbloat
shaper_rate_adjust_up_load_high=1.05 # how rapidly to increase shaper rate upon high load detected
shaper_rate_adjust_down_load_low=0.99 # how rapidly to return down to base shaper rate upon idle or low load detected
shaper_rate_adjust_up_load_low=1.01 # how rapidly to return up to base shaper rate upon idle or low load detected
config_file_check="cake-autorate"
I believe the config file above answers the question
Hum... I think it' s running an old stable version, I couldn't find those log options.
I don't know what change should I do here to comply with Debian architecture...
awk: fatal: cannot open file `/sys/fs/cgroup/services/cake-autorate/cpu.stat' for reading: No such file or directory
)
