My rationale is as I explained above to give a reasonably simple example that can be packaged with sqm somehow, not a fully tricked out 100% solution (I also envision a big well-advertised link to the more complete lua implementation developed here). I consider this to be orthogonal to what is developed here (but still implementing the ideas refined in this thread)
Rate-finding wise, I am quite happy with a simple reactive control loop for my example code. IMHO the example only needs to give 80% and be good enough to be useful, allowing to keep things simpler.
But no matter what I will not have more time to actually work on that any time soon so I will heed the "hold off" advise by default
You guys are awesome thanks so much for all of the work!!! Love this very much lol...
This is the best I can get but lots better than it was. I have Spectrum Business internet docsis 3.1 with 600dl and 35 up. Consistent results on 5g even on congested network...
#!/bin/sh
# automatically adjust bandwidth for CAKE in dependence on detected load and RTT
# inspired by @moeller0 (OpenWrt forum)
# initial sh implementation by @Lynx (OpenWrt forum)
# requires packages: iputils-ping, coreutils-date and coreutils-sleep
debug=1
enable_verbose_output=1 # enable (1) or disable (0) output monitoring lines showing bandwidth changes
ul_if=wan # upload interface
dl_if=ifb4wan # download interface
base_ul_rate=35000 # steady state bandwidth for upload
base_dl_rate=40000 # steady state bandwidth for download
tick_duration=0.5 # seconds to wait between ticks
alpha_RTT_increase=0.001 # how rapidly baseline RTT is allowed to increase
alpha_RTT_decrease=0.9 # how rapidly baseline RTT is allowed to decrease
rate_adjust_RTT_spike=0.01 # how rapidly to reduce bandwidth upon detection of bufferbloat
rate_adjust_load_high=0.005 # how rapidly to increase bandwidth upon high load detected
rate_adjust_load_low=0.0025 # how rapidly to return to base rate upon low load detected
load_thresh=0.5 # % of currently set bandwidth for detecting high load
max_delta_RTT=16 # increase from baseline RTT for detection of bufferbloat
# verify these are correct using 'cat /sys/class/...'
case "${dl_if}" in
\veth*)
rx_bytes_path="/sys/class/net/${dl_if}/statistics/tx_bytes"
;;
\ifb*)
rx_bytes_path="/sys/class/net/${dl_if}/statistics/tx_bytes"
;;
*)
rx_bytes_path="/sys/class/net/${dl_if}/statistics/rx_bytes"
;;
esac
case "${ul_if}" in
\veth*)
tx_bytes_path="/sys/class/net/${ul_if}/statistics/rx_bytes"
;;
\ifb*)
tx_bytes_path="/sys/class/net/${ul_if}/statistics/rx_bytes"
;;
*)
tx_bytes_path="/sys/class/net/${ul_if}/statistics/tx_bytes"
;;
esac
if [ "$debug" ] ; then
echo "rx_bytes_path: $rx_bytes_path"
echo "tx_bytes_path: $tx_bytes_path"
fi
# list of reflectors to use
read -d '' reflectors << EOF
1.1.1.1
8.8.8.8
EOF
RTTs=$(mktemp)
# get minimum RTT across entire set of reflectors
get_RTT() {
for reflector in $reflectors;
do
echo $(/usr/bin/ping -i 0.00 -c 10 $reflector | tail -1 | awk '{print $4}' | cut -d '/' -f 2) >> $RTTs&
done
wait
RTT=$(echo $(cat $RTTs) | awk 'min=="" || $1 < min {min=$1} END {print min}')
> $RTTs
}
call_awk() {
printf '%s' "$(awk 'BEGIN {print '"${1}"'}')"
}
get_next_shaper_rate() {
local cur_delta_RTT
local cur_max_delta_RTT
local cur_rate
local cur_base_rate
local cur_load
local cur_load_thresh
local cur_rate_adjust_RTT_spike
local cur_rate_adjust_load_high
local cur_rate_adjust_load_low
local next_rate
local cur_rate_decayed_down
local cur_rate_decayed_up
cur_delta_RTT=$1
cur_max_delta_RTT=$2
cur_rate=$3
cur_base_rate=$4
cur_load=$5
cur_load_thresh=$6
cur_rate_adjust_RTT_spike=$7
cur_rate_adjust_load_high=$8
cur_rate_adjust_load_low=$9
# in case of supra-threshold RTT spikes decrease the rate so long as there is a load
if awk "BEGIN {exit !(($cur_delta_RTT >= $cur_max_delta_RTT))}"; then
next_rate=$( call_awk "int( ${cur_rate}*(1-${cur_rate_adjust_RTT_spike}) )" )
else
# ... otherwise determine whether to increase or decrease the rate in dependence on load
# high load, so we would like to increase the rate
if awk "BEGIN {exit !($cur_load >= $cur_load_thresh)}"; then
next_rate=$( call_awk "int( ${cur_rate}*(1+${cur_rate_adjust_load_high}) )" )
else
# low load, so determine whether to decay down towards base rate, decay up towards base rate, or set as base rate
cur_rate_decayed_down=$( call_awk "int( ${cur_rate}*(1-${cur_rate_adjust_load_low}) )" )
cur_rate_decayed_up=$( call_awk "int( ${cur_rate}*(1+${cur_rate_adjust_load_low}) )" )
# gently decrease to steady state rate
if awk "BEGIN {exit !($cur_rate_decayed_down > $cur_base_rate)}"; then
next_rate=$cur_rate_decayed_down
# gently increase to steady state rate
elif awk "BEGIN {exit !($cur_rate_decayed_up < $cur_base_rate)}"; then
next_rate=$cur_rate_decayed_up
# steady state has been reached
else
next_rate=$cur_base_rate
fi
fi
fi
echo "${next_rate}"
}
# update download and upload rates for CAKE
function update_rates {
cur_rx_bytes=$(cat $rx_bytes_path)
cur_tx_bytes=$(cat $tx_bytes_path)
t_cur_bytes=$(date +%s.%N)
rx_load=$( call_awk "(8/1000)*(${cur_rx_bytes} - ${prev_rx_bytes}) / (${t_cur_bytes} - ${t_prev_bytes}) * (1/${cur_dl_rate}) " )
tx_load=$( call_awk "(8/1000)*(${cur_tx_bytes} - ${prev_tx_bytes}) / (${t_cur_bytes} - ${t_prev_bytes}) * (1/${cur_ul_rate}) " )
t_prev_bytes=$t_cur_bytes
prev_rx_bytes=$cur_rx_bytes
prev_tx_bytes=$cur_tx_bytes
# calculate the next rate for dl and ul
cur_dl_rate=$( get_next_shaper_rate "$delta_RTT" "$max_delta_RTT" "$cur_dl_rate" "$base_dl_rate" "$rx_load" "$load_thresh" "$rate_adjust_RTT_spike" "$rate_adjust_load_high" "$rate_adjust_load_low")
cur_ul_rate=$( get_next_shaper_rate "$delta_RTT" "$max_delta_RTT" "$cur_ul_rate" "$base_ul_rate" "$tx_load" "$load_thresh" "$rate_adjust_RTT_spike" "$rate_adjust_load_high" "$rate_adjust_load_low")
if [ $enable_verbose_output -eq 1 ]; then
printf "%s;%14.2f;%14.2f;%14.2f;%14.2f;%14.2f;%14.2f;%14.2f;\n" $( date "+%Y%m%dT%H%M%S.%N" ) $rx_load $tx_load $baseline_RTT $RTT $delta_RTT $cur_dl_rate $cur_ul_rate
fi
}
get_baseline_RTT() {
local cur_RTT
local cur_delta_RTT
local last_baseline_RTT
local cur_alpha_RTT_increase
local cur_alpha_RTT_decrease
local cur_baseline_RTT
cur_RTT=$1
cur_delta_RTT=$2
last_baseline_RTT=$3
cur_alpha_RTT_increase=$4
cur_alpha_RTT_decrease=$5
if awk "BEGIN {exit !($cur_delta_RTT >= 0)}"; then
cur_baseline_RTT=$( call_awk "( 1 - ${cur_alpha_RTT_increase} ) * ${last_baseline_RTT} + ${cur_alpha_RTT_increase} * ${cur_RTT} " )
else
cur_baseline_RTT=$( call_awk "( 1 - ${cur_alpha_RTT_decrease} ) * ${last_baseline_RTT} + ${cur_alpha_RTT_decrease} * ${cur_RTT} " )
fi
echo "${cur_baseline_RTT}"
}
# set initial values for first run
get_RTT
baseline_RTT=$RTT;
cur_dl_rate=$base_dl_rate
cur_ul_rate=$base_ul_rate
# set the next different from the cur_XX_rates so that on the first round we are guaranteed to call tc
last_dl_rate=0
last_ul_rate=0
t_prev_bytes=$(date +%s.%N)
prev_rx_bytes=$(cat $rx_bytes_path)
prev_tx_bytes=$(cat $tx_bytes_path)
if [ $enable_verbose_output -eq 1 ]; then
printf "%25s;%14s;%14s;%14s;%14s;%14s;%14s;%14s;\n" "log_time" "rx_load" "tx_load" "baseline_RTT" "RTT" "delta_RTT" "cur_dl_rate" "cur_ul_rate"
fi
# main loop runs every tick_duration seconds
while true
do
t_start=$(date +%s.%N)
get_RTT
delta_RTT=$( call_awk "${RTT} - ${baseline_RTT}" )
baseline_RTT=$( get_baseline_RTT "$RTT" "$delta_RTT" "$baseline_RTT" "$alpha_RTT_increase" "$alpha_RTT_decrease" )
update_rates
# only fire up tc if there are rates to change...
if [ "$last_dl_rate" -ne "$cur_dl_rate" ] ; then
#echo "tc qdisc change root dev ${dl_if} cake bandwidth ${cur_dl_rate}Kbit"
tc qdisc change root dev ${dl_if} cake bandwidth ${cur_dl_rate}Kbit
fi
if [ "$last_ul_rate" -ne "$cur_ul_rate" ] ; then
#echo "tc qdisc change root dev ${ul_if} cake bandwidth ${cur_ul_rate}Kbit"
tc qdisc change root dev ${ul_if} cake bandwidth ${cur_ul_rate}Kbit
fi
# remember the last rates
last_dl_rate=$cur_dl_rate
last_ul_rate=$cur_ul_rate
t_end=$(date +%s.%N)
sleep_duration=$( call_awk "${tick_duration} - ${t_end} + ${t_start}" )
if awk "BEGIN {exit !($sleep_duration > 0)}"; then
sleep $sleep_duration
fi
done
Nice! Would you be willing to run this same test using the experimental branch of the code?
It would help to have another real-world comparison between the two (shell vs Lua). Thanks!
Ooops, forgot to mention… the experimental code does take a little while to “stabilize” and learn your “safe” speeds. So if you get less than ideal tests right away, test it again after maybe 30-60 minutes and see how they look.
It’s still a WIP, but getting real feedback is hugely helpful.
This DRAFT is for brave souls only. It contains my understanding of what we are asking people to do now. I'm still makin' stuff up, so comments are appreciated.
The various implementations are being developed and changed as we speak. We don't promise that any of these control latency, or even that they function at all. They are certainly not nicely packaged, and some assembly is definitely required. If we haven't scared you away...
Does anything interesting happen if you run it long term? (crash, excessive CPU/memory, etc)
If using the lua version, either wait a while (an hour?) for the algorithm to build up some history, or record initial behavior and subsequent history.
Is this list close to useful? What else should we add?
Did we mention that this is a DRAFT? Please send me comments and I'll update. After we have something more suitable/stable, I expect we'll choose a method to publish our experimental results. Thanks.
Changes:
29Dec2021 - point at lua-threads version since it's now good enough to test
This applies to both the experimental branch and dansbranch, LuaJIT or not, both branches are using the same algorithm, they difference is now that experimental is based on coroutines and dansbranch is now using threads
Otherwise I think that's a good starting point yes : )
IMHO, we need to take “dansbranch” out of the mix. It was never intended to be a “stable” test branch for general usage.
We aren’t ready for general multi-threaded testing. I was really only looking for a quick comparison of the algorithms from @Reeves0724 between shell and “experimental” and didn’t mean to open the doors for large scale testing.
@dlakelan and @Lochnair can certainly weigh in here if they have differing opinions.
Sorry at work now my friend will post all the results tonight around 6 p.m. Chicago time... I tested I think all of them that work this Branch gave me the best results... the l u a says jit is missing wish I would have posted that screenshot I will tonight maybe you guys would assist me as I'm just a noob! Also forgive my grammar This is speech to text on the road
No it's still a bit early I agree. Either way I think we should clean up the naming of our branches, so it's more clear which of them are at a point that it's worth trying out at all (like the experimental branch).
The branches we're actively working on are in a constant state of flux, so those tend to break a lot. I'd recommend staying away from those if you don't want to get your hands dirty.
Agreed, it's supposed to be where I tweak rate finding without worrying about breaking anyone's anything :-). We are close to having multithreading testable but I think when that's ready we can branch an testable-mt branch.
For one thing the install script for multithreading will need to pull in additional libraries.
My wan connection is pppoe and Is this what supposed to be? Options are br-lan, eth0, eth1, ifb4eth0, ifb4pppoe-wan, lo, pppoe-wan, teql0, wlan0 in /sys/class/net
I'm getting error:
root@OpenWrt:~# ./autorate.sh
rx_bytes_path: /sys/class/net/br-lan/statistics/rx_bytes
tx_bytes_path: /sys/class/net/pppoe-wan/statistics/tx_bytes
ping: ping: socketsocket: Address family not supported by protocol
: Address family not supported by protocol
log_time; rx_load; tx_load; baseline_RTT; RTT; delta_RTT; cur_dl_rate; cur_ul_rate;
ping: ping: socketsocket: Address family not supported by protocol
: Address family not supported by protocol
20211229T001038.552218451; 0.00; 0.01; 1.94; 1.94; -0.01; 11500.00; 11500.00;
RTNETLINK answers: No such file or directory
ping: ping: socketsocket: Address family not supported by protocol
: Address family not supported by protocol
20211229T001039.046409316; 0.00; 0.00; 1.94; 2.00; 0.06; 11500.00; 11500.00;
ping: ping: socketsocket: Address family not supported by protocol
: Address family not supported by protocol
20211229T001039.553800463; 0.00; 0.00; 1.94; 2.01; 0.07; 11500.00; 11500.00;
Three mentioned packages are installed. Didn't add ipv6 in my firmware since my ISP supports only ipv4. So what's this error "ping: socketsocket: Address family not supported by protocol"?
This is imho a mistake. Ipv6 is here now and needs to be a thing people get used to. The future of openwrt imho is that the easiest and most supported administration method will be ipv6 ULA.
In any case in your specific case I believe In the lua version that we are listening for replies on :: which is an ipv6 address that represents all addresses BOTH ipv6 and ipv4. So I suspect that's causing your problem. Thanks for the report!
Thanks to some quick development work by @Lochnair and @dlakelan over the past few days, the "official" Lua test branch (w/preemptive threading) is ready now. It is the testing/lua-threads branch and here is the README:
There are wget based installation instructions for those who want to try it out without pulling the project repo. It can be installed via: sh -c "$(wget -q -O- https://raw.githubusercontent.com/Fail-Safe/sqm-autorate/testing/lua-threads/sqm-autorate-setup.sh)"
If you do choose to pull the project repo, you can still run the sqm-autorate-setup.sh file within the project and it will install the necessary pieces from the local files. This is useful for testing and tuning purposes.
Within the README are instructions for use, as well as descriptions for the available options and their purposes. Many thanks to @CharlesJC for his work to integrate a native OpenWrt config into this project.
Please refer to this evolving post by @richb-hanover-priv for testing expectations:
Feature/enhancement requests can be submitted via the same URL.
For anyone browsing my repo, please know that the develop/... branches are in no way expected to be stable. Please do not use them if you are not willing to help fix the issues you will uncover. Do NOT open bug reports for develop/... branch code.
The testing/... branches come with a reasonable expectation that obvious runtime bugs have already been squashed. Issues uncovered within testing/... branch code are fair game for bug reports.
Thanks to all who continue to contribute to the discussion and progressions here!
Thanks to @_FailSafe for the final push and cleaning up the repo to make it usable by others. Please do read the readme which has some caveats and expectations. Once this seems to run for people I do think it's fair game for people to fork the repo and try out different ratecontrol functions/algorithms.
If people are getting more speed on few sites like google drive, youtube etc. will the script manage individual ingress bandwidth while maintaining bufferbloat to minimum?
root@RaspberryCM4:~# lua /usr/lib/sqm-autorate/sqm-autorate.lua
lua: error loading module 'lanes.core' from file '/usr/lib/lua/lanes/core.so':
Error relocating /usr/lib/lua/lanes/core.so: pthread_yield: symbol not found
stack traceback:
[C]: ?
[C]: in function 'require'
/usr/lib/lua/lanes.lua:38: in main chunk
[C]: in function 'require'
/usr/lib/sqm-autorate/sqm-autorate.lua:9: in main chunk
[C]: ?
root@RaspberryCM4:~#
(but still implementing the ideas refined in this thread)