Finally, qos-re (originally for WhiteRussian -- see A qos script that really works!) is available for Backfire. The version is bumped up to 2.0.

Install with:

ipkg install http://files.eschauzier.org/qos-re_2.0-1_all.ipk

The changelog:
1. First version for Backfire
2. Removed ipp2p because implementation broken

The hfsc version of the v2.0 script is also available, install with:

ipkg install http://files.eschauzier.org/qos-re-hfsc_2.0-1_all.ipk

The changelog is the same as for qos-re.

To obtain the new /etc/qos.conf file without the ipp2p reference when upgrading, make sure to use the config file that the package manager saves under the name qos.conf-ipkg. Also, you may need to remove existing packages when installing a package with a new name (e.g. qos-re-hfsc over qos-re).

After modifying /etc/qos.conf, you will need to run qos-start to load the new settings. The command qos-stat is available to check the current QoS status. Use qos-stop to stop QoS shaping.

The bare scripts and config file can be found here:
HTB version of the script: 20-qos-htb
HFSC version of the script: 20-qos-hfsc
Config file: qos.conf

Thank you! I will give it a good try.

I have two remarks:
- Could you use a mark mask in the iptable-rules? This make the script more compatible with other apps that use the iptables mark action.
- I have three WAN interfaces. Could you make the script more multi-wan friendly?

Fix the link please

Can you indicate what the problem is? The links seem to be up:

Website test results

URL tested:    http://files.eschauzier.org/qos-re-hfsc_2.0-1_all.ipk
Test performed from:    Seattle, WA
Test performed at:    2012-01-04 15:20:18 (GMT -05:00)
Resolved As:    92.254.60.135
Status:    OK
Response Time:    0.752 sec
DNS:    0.094 sec
Connect:    0.164 sec
Redirect:    0.000 sec
First byte:    0.281 sec
Last byte:    0.213 sec
Size:    7856 bytes

If I remember correctly, Black Russian had two qos implementations. One of them became stock implementation. This is the upgrade of another one, correct?

So, in order to use this, I need to uninstall qos-scripts, right? Is it compatible with luci UI? In other words, if I install qos-re, will I be able to administer it through web UI?

I also remember a spirited discussion (to detailed for me to understand ) about the differences between the two. Now my bandwidth got bigger (and my son got older, so he is not saturating network with file sharing as before) - but is there a way to see performance difference differences between two versions?

Thanks

Rudy,

Fist: I’m neither a linux guru nor a network guru. And my English is very bad but maybe you and other people could use my experience.

I’ve written my own QoS script because I have a slow ADSL connection (6144kbit / 640kbit) and I would prioritize HTTP and VoIP. The default OpenWRT script isn’t documented, Service curves are wrong calculated (in my vieuw) and it isn’t possible to configure the leaf Qdisc. Tomato doesn’t use a good download QoS. The leaf Qdisc of DD-wrt also aren’t configurable.

Instead of writing your own script, it’s maybe possible to change the default OpenWrt script and make some documentation? If there are some good Qos configfiles with a good documentation, it isn’t necessary to write another script. I’m not capable to modify the default QoS script because I’m not a linux programmer. I understand most of the default QoS script but as already told I’m not capable to modify the default QoS script. Maybe you can change the QoS script of nbd? I would like to help make some wiki documentation (i know people who will translate in English)

Change requests rudy’s and nbd’s QoS script
1) Use the TC stab option if you use the thrunk version of OpenWrt. It really works With the stab option, TC calculate the (ATM, PPP,…) overhead. So it’s not necessary to set upload 90% of your linespeed.

tc qdisc add dev $iface stab overhead 44 linklayer atm root handle 1: hfsc default 50

I guess, it's not difficult to implement the stab option in the default QoS script. Just add 2 configrules and some code.
suggestion of the extra configrules:
option linklayer_adaptation "ATM" or "ethernet"
option overhead "44" (per ethernet packet 44 bytes overhead) link

2) The default QoS of the thrunk version of OpenWrt use a ack_conntrack patch and the IFB interface. Maybe you could use IFB and act_conntrack?

3) In your script you say not to use “length of burst buffers lower than 10 ms) => timer resolution is 100HZ. Is it possible to find out the timer resolution of the kernel? I don’t know which timer resolution openwrt use. A cat of the /proc/timer_list shows me a timer resolution of 1ns (WNDR 3700v2)

/proc$ cat /proc/timer_list
Timer List Version: v0.6
HRTIMER_MAX_CLOCK_BASES: 3
now at 785328994524672 nsecs

cpu: 0
 clock 0:
  .base:       802c7fa0
  .index:      0
  .resolution: 1 nsecs
  .get_time:   <80095004>
  .offset:     1326047247244582392 nsecs
active timers:
 clock 1:
  .base:       802c7fd0
  .index:      1
  .resolution: 1 nsecs
  .get_time:   <800952e0>
  .offset:     0 nsecs
active timers:
 #0: <8399fb98>, <801f20dc>, S:01
 # expires at 785328994924288-785328994924288 nsecs [in 399616 to 399616 nsecs]
 #1: <830ff798>, <801f20dc>, S:01
 # expires at 785328996922816-785328996922816 nsecs [in 2398144 to 2398144 nsecs]
 #2: <8030b6c0>, <800700ec>, S:01
 # expires at 785329000000000-785329000000000 nsecs [in 5475328 to 5475328 nsecs]
 #3: <802c84b0>, <8009ac40>, S:01
 # expires at 785329000000000-785329000000000 nsecs [in 5475328 to 5475328 nsecs]
 #4: <8381deb8>, <80090244>, S:01
 # expires at 785329047731730-785329047781730 nsecs [in 53207058 to 53257058 nsecs]
 #5: <83355a48>, <80090244>, S:01
 # expires at 785329074610491-785329075568298 nsecs [in 80085819 to 81043626 nsecs]
 #6: <8385deb8>, <80090244>, S:01
 # expires at 785330038946170-785330038996170 nsecs [in 1044421498 to 1044471498 nsecs]
 #7: <83203a48>, <80090244>, S:01
 # expires at 785336390075945-785336400075938 nsecs [in 7395551273 to 7405551266 nsecs]
 #8: <82569a48>, <80090244>, S:01
 # expires at 785628990456272-785629090456272 nsecs [in 299995931600 to 300095931600 nsecs]
 #9: <83351ae0>, <80090244>, S:01
 # expires at 786283993381947-786284093381947 nsecs [in 954998857275 to 955098857275 nsecs]
 #10: <83a31e50>, <80079598>, S:01
 # expires at 811277106541532-811277106541532 nsecs [in 25948112016860 to 25948112016860 nsecs]
 clock 2:
  .base:       802c8000
  .index:      7
  .resolution: 1 nsecs
  .get_time:   <80094d08>
  .offset:     0 nsecs
active timers:
  .expires_next   : 785328994924288 nsecs
  .hres_active    : 1
  .nr_events      : 154691328
  .nr_retries     : 486291
  .nr_hangs       : 1
  .max_hang_time  : 17803 nsecs
  .nohz_mode      : 0
  .idle_tick      : 0 nsecs
  .tick_stopped   : 0
  .idle_jiffies   : 0
  .idle_calls     : 0
  .idle_sleeps    : 0
  .idle_entrytime : 0 nsecs
  .idle_waketime  : 0 nsecs
  .idle_exittime  : 0 nsecs
  .idle_sleeptime : 0 nsecs
  .iowait_sleeptime: 0 nsecs
  .last_jiffies   : 0
  .next_jiffies   : 0
  .idle_expires   : 0 nsecs
jiffies: 78502899


Tick Device: mode:     1
Per CPU device: 0
Clock Event Device: MIPS
 max_delta_ns:   6316128371
 min_delta_ns:   2258
 mult:           1460288881
 shift:          32
 mode:           3
 next_event:     785328994924288 nsecs
 set_next_event: <800684f0>
 set_mode:       <80068514>
 event_handler:  <80090e9c>
 retries:        1

4) it should be possible to set each parameter of the HFSC class. Maybe this is possible in the default openwrt QoS script but I have not studied the entire script.

5) it should be possible to set each parameter of the leaf qdisc. The most home computer do not support ECN by default. I also thing the most webservers don't support ECN.  So it should be possible to disable ECN.
It’s possible to limit P2P traffic with a large leaf red qdisc (300 ms). I’ve read that P2P congestion control algorithms take RTT of the packets into account. In my case, it works . No packets are dropped and the RTT of ICMP packets are very good. So make it possible to the user to change the leaf qdisc parameters.
I test this situation by sending ICMP packets to the priority class. the RTT of the ICMP packets are good. RTT of ICMP packets while downloading HTTP content (youtube, vimeo,...) are extremely good.

My script (based on rudy’s and nbd’s one):

#!/bin/sh -x

#qos script by TomVH

DEBUG=0

# To enable logging (requires iptables-mod-extra package)
[ $DEBUG -eq 1 ] && insmod ipt_LOG >&- 2>&-

#######################################################
DOWNLOAD=5000 #download speed in kbit. set xx% of real download speed
UPLOAD=600 # set xx% of real upload speed

# multiports = up to 15 ports
# ports to be classified as bulk #set after connection mark save and after connection mark restore
TCP_BULK="1024:" #S and D ports
UDP_BULK="1024:" #S and D ports

# Destination ports to be classified as P2P
TCP_P2P="13769" #D ports
UDP_P2P="13769" #D ports
IP_P2P="192.168.0.133"

# Destination ports to be classified as normal
TCP_NORMAL="80,443,25,20,21,110,993,995" # D ports
UDP_NORMAL=""

# Destination ports to be classified as Prio (overules bulk ports)
TCP_PRIO="22,53" #destination ports
UDP_PRIO="22,53" #destination ports

# Destination ports to be classified as VoIP (overules bulk ports)
TCP_VOIP=""
UDP_VOIP="18080"
IP_VOIP="192.168.0.226" #destination and source IP
IP_VOIP="192.168.0.226" #destination and source IP

#######################################################
iface="pppoe-wan"
#####################################################


#!!!!!uplink leaf class parameters!!!!!!!!!

#bulk
UP_LS_BULK_RATE=$(($UPLOAD*5/100))
UP_UL_BULK_RATE=$UPLOAD
#settings leaf qdisc
UP_BULK_RED_PROB=0.05 #red drob probability
UP_BULK_RED_min=6250 #real limit. To limit BULK traffic
UP_BULK_RED_min2=6250 #min for doing the calculations (burst and etc)
UP_BULK_RED_max=$((2 * $UP_BULK_RED_min2 + $UP_BULK_RED_min))
UP_BULK_RED_burst=$(((5 * $UP_BULK_RED_min2) / (3 * 1000)))
UP_BULK_RED_limit=$(($UP_BULK_RED_max * 5))

#P2P
UP_LS_P2P_RATE=$(($UPLOAD * 5 / 100))
UP_UL_P2P_RATE=$UPLOAD
#settings leaf qdisc
UP_P2P_RED_PROB=0.05 #red drob probability
UP_P2P_RED_min=32000 #real limit. To limit P2P traffic
UP_P2P_RED_min2=32000 #min for doing the calculations (burst and etc)
UP_P2P_RED_max=$((5 * $UP_P2P_RED_min2 + $UP_P2P_RED_min))
UP_P2P_RED_burst=$(((5 * $UP_P2P_RED_min2) / (3 * 1000)))
UP_P2P_RED_limit=$(($UP_P2P_RED_max * 5))
#normal class
UP_LS_NORMAL_RATE=$(($UPLOAD * 40 / 100))
UP_UL_NORMAL_RATE=$UPLOAD
#settings leaf qdisc
UP_NORMAL_RED_PROB=0.05 #red drob probability
UP_NORMAL_RED_min=6250 #real limit. To limit NORMAL traffic
UP_NORMAL_RED_min2=6250 #min for doing the calculations (burst and etc)
UP_NORMAL_RED_max=$((2 * $UP_NORMAL_RED_min2 + $UP_NORMAL_RED_min))
UP_NORMAL_RED_burst=$(((5 * $UP_NORMAL_RED_min2) / (3 * 1000)))
UP_NORMAL_RED_limit=$(($UP_NORMAL_RED_max * 5))

#prio
UP_LS_PRIO_RATE=$(($UPLOAD*50/100))
UP_RT_PRIO_RATE="200" #rate in kbit
UP_RT_PRIO_UMAX="400" #lengte of the packets [byte]
UP_RT_PRIO_DMAX="15" #delay in ms
UP_UL_PRIO_RATE=$UPLOAD

#Voip
UP_UL_VOIP_RATE=$UPLOAD
UP_SC_VOIP_RATE="200"
UP_SC_VOIP_UMAX="350" #length of the voip packets [byte]
UP_SC_VOIP_DMAX="10" #delay in ms



#!!!!!DOWNLIK leaf class parameters!!!!!!!!!

#bulk
DOWN_LS_BULK_RATE=$(($DOWNLOAD*5/100))
DOWN_UL_BULK_RATE=$DOWNLOAD
#leaf qdisc parameters
DOWN_BULK_RED_PROB=0.05 #red drob probability
DOWN_BULK_RED_min=62500 #real limit. To limit BULK traffic
DOWN_BULK_RED_min2=62500 #min for doing the calculations (burst and etc)
DOWN_BULK_RED_max=$((2 * $DOWN_BULK_RED_min2 + $DOWN_BULK_RED_min))
DOWN_BULK_RED_burst=$(((5 * $DOWN_BULK_RED_min2) / (3 * 1000)))
DOWN_BULK_RED_limit=$(($DOWN_BULK_RED_max * 5))


#P2P
DOWN_LS_P2P_RATE=$(($DOWNLOAD*5/100))
DOWN_UL_P2P_RATE=4000
#leaf qdisc parameters
DOWN_P2P_RED_PROB=0.05 #red drob probability
DOWN_P2P_RED_min=200000 #real limit. To limit P2P traffic
DOWN_P2P_RED_min2=200000 #min for doing the calculations (burst and etc)
DOWN_P2P_RED_max=$((2 * $DOWN_P2P_RED_min2 + $DOWN_P2P_RED_min))
DOWN_P2P_RED_burst=$(((5 * $DOWN_P2P_RED_min2) / (3 * 1000)))
DOWN_P2P_RED_limit=$(($DOWN_P2P_RED_max * 5))

#normal class
DOWN_LS_NORMAL_RATE=$(($DOWNLOAD*75/100))
DOWN_UL_NORMAL_RATE=$DOWNLOAD

#leaf qdisc parameters
DOWN_NORMAL_RED_PROB=0.05 #red drob probability
DOWN_NORMAL_RED_min=62500 #real limit. To limit NORMAL traffic
DOWN_NORMAL_RED_min2=62500 #min for doing the calculations (burst and etc)
DOWN_NORMAL_RED_max=$((2 * $DOWN_NORMAL_RED_min2 + $DOWN_NORMAL_RED_min))
DOWN_NORMAL_RED_burst=$(((5 * $DOWN_NORMAL_RED_min2) / (3 * 1000)))
DOWN_NORMAL_RED_limit=$(($DOWN_NORMAL_RED_max * 5))


#prio
DOWN_RT_PRIO_RATE="500" #rate in kbit
DOWN_RT_PRIO_UMAX="400" #length of the packets [byte]
DOWN_RT_PRIO_DMAX="1.5" #delay in ms
DOWN_UL_PRIO_RATE=$DOWNLOAD


#Voip
DOWN_UL_VOIP_RATE=$DOWNLOAD
DOWN_SC_VOIP_RATE="250"
DOWN_SC_VOIP_UMAX="350" #lengt of voip packets [byte]
DOWN_SC_VOIP_DMAX="1.2" #delay in ms


# The following packages are required for the modules:
# kmod-sched
# kmod-ipt-conntrack
# iptables-mod-conntrack
# kmod-ipt-ipopt
# iptables-mod-ipopt
# kmod-ipt-extra
# iptables-mod-extra

insmod ifb
insmod sch_hfsc
insmod sch_red
insmod sch_sfq
insmod cls_fw
insmod cls_u32
insmod em_u32
insmod act_connmark
insmod act_mirred
insmod sch_ingress

tc qdisc del dev $iface ingress
tc qdisc add dev $iface ingress
ifconfig ifb0 up txqueuelen 5
tc filter add dev $iface parent ffff: protocol ip prio 1 u32 match u32 0 0 flowid 1:1 action connmark action mirred egress redirect dev ifb0

#ecn isn't supported by default in win7 winXp,... So do not enable ECN of the uplaod qdiscs!!!
ifconfig $iface up txqueuelen 5
tc qdisc add dev $iface stab overhead 44 linklayer atm root handle 1: hfsc default 50 
tc class add dev $iface parent 1: classid 1:1 hfsc sc rate ${UPLOAD}kbit ul rate ${UPLOAD}kbit
tc class add dev $iface parent 1:1 classid 1:10 hfsc sc umax ${UP_SC_VOIP_UMAX}b dmax ${UP_SC_VOIP_DMAX}ms rate ${UP_SC_VOIP_RATE}kbit ul rate ${UP_UL_VOIP_RATE}kbit
tc class add dev $iface parent 1:1 classid 1:20 hfsc rt umax ${UP_RT_PRIO_UMAX}b dmax ${UP_RT_PRIO_DMAX}ms rate ${UP_RT_PRIO_RATE}kbit ls rate ${UP_LS_PRIO_RATE}kbit  ul rate ${UP_UL_PRIO_RATE}kbit
tc class add dev $iface parent 1:1 classid 1:30 hfsc ls rate ${UP_LS_NORMAL_RATE}kbit  ul rate ${UP_UL_NORMAL_RATE}kbit
tc class add dev $iface parent 1:1 classid 1:40 hfsc ls rate ${UP_LS_P2P_RATE}kbit  ul rate ${UP_UL_P2P_RATE}kbit
tc class add dev $iface parent 1:1 classid 1:50 hfsc ls rate ${UP_LS_BULK_RATE}kbit  ul rate ${UP_UL_BULK_RATE}kbit
tc qdisc add dev $iface parent 1:10 handle 100: sfq perturb 2 #limit xxx =>128 is the default limit
tc qdisc add dev $iface parent 1:20 handle 200: sfq perturb 2 #limit xxx =>128 is the default limit
tc qdisc add dev $iface parent 1:30 handle 300: red limit $UP_NORMAL_RED_limit min $UP_NORMAL_RED_min max $UP_NORMAL_RED_max avpkt 1000 burst $UP_NORMAL_RED_burst probability $UP_NORMAL_RED_PROB 
tc qdisc add dev $iface parent 1:40 handle 400: red limit $UP_P2P_RED_limit min $UP_P2P_RED_min max $UP_P2P_RED_max avpkt 1000 burst $UP_P2P_RED_burst probability $UP_P2P_RED_PROB
tc qdisc add dev $iface parent 1:50 handle 500: red limit $UP_BULK_RED_limit min $UP_BULK_RED_min max $UP_BULK_RED_max avpkt 1000 burst $UP_BULK_RED_burst probability $UP_BULK_RED_PROB 
tc filter add dev $iface parent 1: prio 1 protocol ip handle 1 fw flowid 1:10
tc filter add dev $iface parent 1: prio 2 protocol ip handle 2 fw flowid 1:20
tc filter add dev $iface parent 1: prio 3 protocol ip handle 3 fw flowid 1:30
tc filter add dev $iface parent 1: prio 4 protocol ip handle 4 fw flowid 1:40
tc filter add dev $iface parent 1: prio 5 protocol ip handle 5 fw flowid 1:50

tc qdisc add dev ifb0 stab overhead 44 linklayer atm root handle 1: hfsc default 50
tc class add dev ifb0 parent 1: classid 1:1 hfsc sc rate ${DOWNLOAD}kbit ul rate ${DOWNLOAD}kbit
tc class add dev ifb0 parent 1:1 classid 1:10 hfsc sc umax ${DOWN_SC_VOIP_UMAX}b dmax ${DOWN_SC_VOIP_DMAX}ms rate ${DOWN_SC_VOIP_RATE}kbit ul rate ${DOWN_UL_VOIP_RATE}kbit
tc class add dev ifb0 parent 1:1 classid 1:20 hfsc sc umax ${DOWN_RT_PRIO_UMAX}b dmax ${DOWN_RT_PRIO_DMAX}ms rate ${DOWN_RT_PRIO_RATE}kbit ul rate ${DOWN_UL_PRIO_RATE}kbit
tc class add dev ifb0 parent 1:1 classid 1:30 hfsc ls rate ${DOWN_LS_NORMAL_RATE}kbit  ul rate ${DOWN_UL_NORMAL_RATE}kbit
tc class add dev ifb0 parent 1:1 classid 1:40 hfsc ls rate ${DOWN_LS_P2P_RATE}kbit  ul rate ${DOWN_UL_P2P_RATE}kbit
tc class add dev ifb0 parent 1:1 classid 1:50 hfsc ls rate ${DOWN_LS_BULK_RATE}kbit  ul rate ${DOWN_UL_BULK_RATE}kbit
tc qdisc add dev ifb0 parent 1:10 handle 100: sfq perturb 2 #limit xxx =>128 is the default limit
tc qdisc add dev ifb0 parent 1:20 handle 200: sfq perturb 2 #limit xxx =>128 is the default limit
tc qdisc add dev ifb0 parent 1:30 handle 300: red limit $DOWN_NORMAL_RED_limit min $DOWN_NORMAL_RED_min max $DOWN_NORMAL_RED_max avpkt 1000 burst $DOWN_NORMAL_RED_burst probability $DOWN_NORMAL_RED_PROB
tc qdisc add dev ifb0 parent 1:40 handle 400: red limit $DOWN_P2P_RED_limit min $DOWN_P2P_RED_min max $DOWN_P2P_RED_max avpkt 1000 burst $DOWN_P2P_RED_burst probability $DOWN_P2P_RED_PROB
tc qdisc add dev ifb0 parent 1:50 handle 500: red limit $DOWN_BULK_RED_limit min $DOWN_BULK_RED_min max $DOWN_BULK_RED_max avpkt 1000 burst $DOWN_BULK_RED_burst probability $DOWN_BULK_RED_PROB
tc filter add dev ifb0 parent 1: prio 1 protocol ip handle 1 fw flowid 1:10
tc filter add dev ifb0 parent 1: prio 2 protocol ip handle 2 fw flowid 1:20
tc filter add dev ifb0 parent 1: prio 3 protocol ip handle 3 fw flowid 1:30
tc filter add dev ifb0 parent 1: prio 4 protocol ip handle 4 fw flowid 1:40
tc filter add dev ifb0 parent 1: prio 5 protocol ip handle 5 fw flowid 1:50



##########################
#####    IPTABLES    #####
##########################

iptables -t mangle -F QOS
iptables -t mangle -D FORWARD -o $iface -j QOS
iptables -t mangle -D OUTPUT -o $iface -j QOS
iptables -t mangle -X QOS

insmod ipt_layer7
insmod xt_layer7
insmod ipt_connbytes
insmod ipt_tos
insmod ipt_dscp
insmod ipt_length
insmod ipt_CONNMARK
insmod ipt_multiport

iptables -t mangle -N QOS
iptables -t mangle -A FORWARD -o $iface -j QOS
iptables -t mangle -A OUTPUT -o $iface -j QOS

#restore connection mark
iptables -t mangle -A QOS -j CONNMARK --restore-mark

#mark everything before connection mark store
#ICMP packages must by prio
iptables -t mangle -A QOS -s 192.168.0.145 -p icmp -j MARK --set-mark 2

#VoIP packages must be marked
iptables -t mangle -A QOS -p udp -m multiport --sports ${UDP_VOIP} -j MARK --set-mark 1
iptables -t mangle -A QOS -s ${IP_VOIP} -j MARK --set-mark 1
iptables -t mangle -A QOS -d ${IP_VOIP} -j MARK --set-mark 1
#may marked if not yet marked
iptables -t mangle -A QOS -m mark --mark 0 -p tcp -m multiport --dports ${TCP_PRIO} -j MARK --set-mark 2
iptables -t mangle -A QOS -m mark --mark 0 -p udp -m multiport --dports ${UDP_PRIO} -j MARK --set-mark 2
iptables -t mangle -A QOS -m mark --mark 0 -p tcp -m multiport --dports ${TCP_NORMAL} -j MARK --set-mark 3

#P2P traffic
iptables -t mangle -A QOS -s ${IP_P2P} -m mark --mark 0 -p tcp -m multiport --sports ${TCP_P2P} -j MARK --set-mark 4
iptables -t mangle -A QOS -s ${IP_P2P} -m mark --mark 0 -p udp -m multiport --sports ${UDP_P2P} -j MARK --set-mark 4

#bulk traffiek
iptables -t mangle -A QOS -m mark --mark 0 -p tcp --sport ${TCP_BULK} -j MARK --set-mark 5
iptables -t mangle -A QOS -m mark --mark 0 -p udp --sport ${UDP_BULK} -j MARK --set-mark 5
iptables -t mangle -A QOS -m mark --mark 0 -p tcp --dport ${TCP_BULK} -j MARK --set-mark 5
iptables -t mangle -A QOS -m mark --mark 0 -p udp --dport ${UDP_BULK} -j MARK --set-mark 5

#alles moet een mark 3 "normal" krijgen als ze nog niet gemarked zijn. Dit is de enige manier om utorrent te snoeren. By deafault moeten alle pakketen in de P2P class behalve als ze gemarked zijn.
iptables -t mangle -A QOS -m mark --mark 0 -j MARK --set-mark 3


# Save mark onto connection
iptables -t mangle -A QOS -j CONNMARK --save-mark

#reclasify any packets #zeker nog te bekijken welke flags we moeten zetten!!
#iptables -t mangle -A QOS -p tcp -m length --length :128 -m mark ! --mark 4 -m mark ! --mark 5 -m tcp --tcp-flags SYN,RST,ACK ACK -j MARK --set-mark 2

#debugging

[ $DEBUG -eq 1 ] && iptables -t mangle -A mark_chain -m mark --mark 0 -j LOG --log-prefix mark_0::
[ $DEBUG -eq 1 ] && iptables -t mangle -A mark_chain -m mark --mark 0 -j ACCEPT
[ $DEBUG -eq 1 ] && iptables -t mangle -A mark_chain -m mark --mark 1 -j LOG --log-prefix mark_1::
[ $DEBUG -eq 1 ] && iptables -t mangle -A mark_chain -m mark --mark 1 -j ACCEPT
[ $DEBUG -eq 1 ] && iptables -t mangle -A mark_chain -m mark --mark 2 -j LOG --log-prefix mark_2::
[ $DEBUG -eq 1 ] && iptables -t mangle -A mark_chain -m mark --mark 2 -j ACCEPT
[ $DEBUG -eq 1 ] && iptables -t mangle -A mark_chain -m mark --mark 3 -j LOG --log-prefix mark_3::
[ $DEBUG -eq 1 ] && iptables -t mangle -A mark_chain -m mark --mark 3 -j ACCEPT
[ $DEBUG -eq 1 ] && iptables -t mangle -A mark_chain -j LOG --log-prefix mark_other::

[ $DEBUG -eq 1 ] && iptables -t mangle -A ingress_chain -m mark --mark 0 -j LOG --log-prefix ingress_0::
[ $DEBUG -eq 1 ] && iptables -t mangle -A ingress_chain -m mark --mark 0 -j ACCEPT
[ $DEBUG -eq 1 ] && iptables -t mangle -A ingress_chain -m mark --mark 1 -j LOG --log-prefix ingress_1::
[ $DEBUG -eq 1 ] && iptables -t mangle -A ingress_chain -m mark --mark 1 -j ACCEPT
[ $DEBUG -eq 1 ] && iptables -t mangle -A ingress_chain -m mark --mark 2 -j LOG --log-prefix ingress_2::
[ $DEBUG -eq 1 ] && iptables -t mangle -A ingress_chain -m mark --mark 2 -j ACCEPT
[ $DEBUG -eq 1 ] && iptables -t mangle -A ingress_chain -m mark --mark 3 -j LOG --log-prefix ingress_3::
[ $DEBUG -eq 1 ] && iptables -t mangle -A ingress_chain -m mark --mark 3 -j ACCEPT
[ $DEBUG -eq 1 ] && iptables -t mangle -A ingress_chain -j LOG --log-prefix ingress_other::

Qdisc hierarchy

(Last edited by TomVH on 17 Jan 2012, 21:38)

yes

yes

I think rudy's script isn't compatible with luci UI. I'm quite sure.

check the RTT of ICMP packets in the prio class?
I don't know if this is a good method but in my case it works well. I have 2 PC's. ICMP packets of 1 PC are prio packets and ICMP packets of the other PC are bulk, normal or P2P packets. The difference is amazing . (in my script but probably it's also possible in the other scripts).

sorry for my bad English.

TomVH, your English is fine... don't worry!

When I asked about performance difference, I meant difference between this script and stock script. When you write about checking the RTT of ICMP packets in the prio class - you probably mean the difference between having qos script and not having qos script - right? or the difference between different packets affected by qos script.

Either way, that doesn't help me to answer the question "how is this script better than the stock script".

Hello rudy, thanks for sharing your work - it is realy useful. I initially discovered your thread in the WhiteRussian section before and was really glad to see your backfire port.

I searched for a QoS solution for my network and started working on my own script that turned into an alternative QoS architecture. It is inspired by your script and shares several common features with it.
Right now, I am at the moment of  testing it and will share the results soon - hope that all of you may find some useful ideas implemented in it. I strive to do it highly customizable and comprehensive. One of the main ideas is to have a multiple QoS interfaces support with different configurations for each option per interface - separate bandwidth, priority, etc.
Hope to finish testing successfully, probably I wiil create a new topic.

Hope that my solution will work in your case.

(Last edited by dir2cas on 18 Jan 2012, 01:02)

You can test both. I explained how you can test both scripts.

Witch QoS script is the best? That is a difficult question. I think the stock script is really good but the implementation of the overhead calculation and configurable leaf qdiscs are really necessary.

Does anyone know how to ask change requests to the nbd’s QoS script? The ticket system?

Write your own script or modify the stock script (nbd's one). Why:

1)    rudy's and nbd's script don't use PPP and ATM overhead calculation. VoIP AND P2P use small Ethernet packets. The overhead (PPP, ATM headers) are not countered by TC by default. So it’s important to use the stab option because overhead takes a part of your bandwidth. If you use VoIP and P2P the overhead can be 20% (or more) of your total available bandwidth. Use wireshark to determine the packet-length of some applications.

2)    Nbd’s script uses the ECN option of the RED leaf qdiscs but I think most of the internet servers don’t support ECN. Win7, ubuntu, WinXP, Win2k don’t support ECN by default. So don’t use ECN.

3)    Rudy’s script use RED leaf qdiscs. He calculates de “min” value by “5*MTU” and mtu=1600byte. If you have a downloadspeed of 5000kbit then t=5*1600byte*8bit/5000kbit=12 ms. I think A queue length of 12 ms is to low. To many packets are dropped. I use a default queue length of 100 ms and is works great with http (vimeo, youtube) traffic. I use a queue length of 300 ms voor P2P (µtorrent) and RTT of prioritized packets are good. Smaller queues results in larger ICMP RTT (probably because to many packets are dropped and the P2P congestion algorithm reacts wrong). You can use small queues in the UPLOAD part but in the download qdiscs I think it’s better to use a longer queue in leaf qdisc. Especially a large queue for µtorrent seems to work well. Maybe there are other people who could test the effect of the queue length with P2P applications? Non of the QoS script use different queue lengths for the different classes. Nbd’s one use a queue length of 50 ms for non prio/express classes. Gargoyle use leaf queues of 200 ms.

4)    Rudy use the term “burst” in HFSC. In my view, the leaf qdisc take care of the burst and not the HFSC algorithm. HFSC is used to reduce delay.

5)    Rudy don’t use the linkshare criterium. He only uses “SC” service curves. (SC combines a linkshare (LS) service cure and a realtime (RT) service curve). If every class starts sending at once, HFSC exceeds the max linespeed!! It’s not necessary to use RT service curves for all classes. Just use RT service curves to the prio classes (VoIP, gaming,…) and use LS service curves for the other classes.

If you want some good links to study HFSC and iptables in linux, ask me and i will post some good links to start studying QoS, HFSC, RED,...

Hello rudy, TomVH and all interested,

Did you manage to find a solution for your script for this issue? Since I examined your code and found that you are also jumping to IMQ from INPUT and FORWARD, which is causing the traffic destined to the local process (router itself) not to be handled by IMQ and therefore not classified. I notised that while testing my architecture that I am building now (as I mentioned).

Example - we have a router generated downlink traffic - let's say a wget downloading a lagre file on the router (or to /dev/null for testing) filling your bandwidth. I noticed that this traffic is passing all marking rules OK and is going (jumping) to the IMQ device as well (I can see counters accumulating in iptables -t mangle -vS) but this is not handled by the imq0 interface itself - its counters remaining unchanged. This causes the traffic not to be passed to the interface qdiscs and therefore it is not classified (tc -s -d class/qdisc show dev imq0  also does not detect activity).

I am working on finding a solution probably using another chain on PREROUTING stage in which will only try to restore the previously set CONNMARK, relaying on the egress marking of a connection(session). This is a similar concept to that suggested in the link above (gargoyle). Even if this fixes the problem we will still be unable to catch non connection-oriented traffic dedicated to the router.

Did you also notice this issue, if yes, what is your solution. If I succeeded I will update you as well.

Regards,

(Last edited by dir2cas on 23 Jan 2012, 20:59)

-You have to mark only outgoing packets.
-So, use the output (router traffic) and forward (LAN traffic) chain to do this
-Use the "save mark" connmark target to save a packet mark tot the connection mark.

If you only allow established and related traffic (default firewall rule) on your network, than I see no problem. Non established or Related packets are dropped by default!!.

Read "chapter 7. the state machine" of the IPtables tutorial to understand it.

Now we don't now when the connection mark is set but I think it’s before the mangle prerouting chain. Probably you can use the "netfilter hooks" in google to find the answer.

In my case, ICMP packets from IP 192.168.0.145 are send to class 200: and it works!!. I'm sure.

I use the trunk version and i think the trunk version of OpenWrt doesn't support IMQ anymore. ndb has written iptables connmark_act patch to restore the connection marks before the packets enter the IFB device.

I see only one problem: broadcast traffic (IPTV). Maybe it's possible to mark incoming iptv packets by the restore-mark connmark action. I don't know because i have no IPTV applications.

I take the offer!

Do you want to work on OpenWrt - QoS (Quality of Service)?

In fact I managed to cope with the local processes traffic (destined to the router itself), by using a little different concept, than the strategy I used for traffic flow marking till now.
Egress: inspected in:
OUTPUT chain (for local processes traffic)
FORWARD chain (for traffic routed though the device), generated from the LAN

Ingress : inspected in:
PREROUTING - next the traffic is thrown in the IMQ device (I use it only once here)

Combined with the proper CONNMARK rules and a optimizing the packet inspection flow through my QoS policy, the desired result was achieved - even with a little less CPU utilization than the default qos-scripts package.
Now both forwarded and router's traffic is properly classified.

Well, all is right, but while testing my solution today I noticed another interesting issue. All packets get classified correctly  - I mean properly matched by in my iptables rules via the Qos Policy and after that properly classified.
But, when the bandwidth limit configured by me for the classes (I used bandwidht sharing strategy) is greater than the currently available through my ISP, the traffic is not handled right despite the fact that it is classified correclt. It seems that each class sees it has more bandwith (tokens available) and none of the classes gets a priority over the others correctly - it can be seen as the ping delay remains high despite it is classified with highest priority.

I can see that 2x Wget processes downloading a large files, together with my IPTV and icmp test are running but none gets a real preference over the others. When I set my BW to be equal or less than the currently available one, all looks rights and goes smoothly. I am sure all traffic get correct marking and priority

[root@Core-Sf:~]# tc -s -d class show dev imq0
class htb 1:1 root rate 26128Kbit ceil 26128Kbit burst 146970b/8 mpu 0b overhead 0b cburst 146970b/8 mpu 0b overhead 0b level 7 
 Sent 132665411 bytes 88786 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 22144Kbit 1853pps backlog 0b 0p requeues 0 
 lended: 51370 borrowed: 0 giants: 0
 tokens: 695669 ctokens: 695669

class htb 1:10 parent 1:1 leaf 10: prio 1 quantum 1522 rate 2048Kbit ceil 26128Kbit burst 48989b/8 mpu 0b overhead 0b cburst 48990b/8 mpu 0b overhead 0b level 0 
 Sent 829463 bytes 885 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 139024bit 19pps backlog 0b 0p requeues 0 
 lended: 885 borrowed: 0 giants: 0
 tokens: 2739399 ctokens: 225678

class htb 1:20 parent 1:1 leaf 20: prio 3 quantum 1522 rate 1024Kbit ceil 26128Kbit burst 48989b/8 mpu 0b overhead 0b cburst 48990b/8 mpu 0b overhead 0b level 0 
 Sent 43276500 bytes 28851 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 8075Kbit 673pps backlog 0b 0p requeues 0 
 lended: 3863 borrowed: 24988 giants: 0
 tokens: -141752 ctokens: 223089

class htb 1:30 parent 1:1 leaf 30: prio 6 quantum 1522 rate 512000bit ceil 23515Kbit burst 48989b/8 mpu 0b overhead 0b cburst 48984b/8 mpu 0b overhead 0b level 0 
 Sent 42405948 bytes 28281 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 6315Kbit 526pps backlog 0b 0p requeues 0 
 lended: 1984 borrowed: 26297 giants: 0
 tokens: -32543 ctokens: 252422

class htb 1:40 parent 1:1 leaf 40: prio 2 quantum 1522 rate 12288Kbit ceil 26128Kbit burst 48987b/8 mpu 0b overhead 0b cburst 48990b/8 mpu 0b overhead 0b level 0 
 Sent 46153500 bytes 30769 pkt (dropped 0, overlimits 0 requeues 0) 
 rate 7616Kbit 635pps backlog 0b 0p requeues 0 
 lended: 30684 borrowed: 85 giants: 0
 tokens: 23583 ctokens: 226616

[root@Core-Sf:~]# tc -s -d filter show dev imq0
filter parent 1: protocol ip pref 1 fw 
filter parent 1: protocol ip pref 1 fw handle 0xa classid 1:10 
filter parent 1: protocol ip pref 2 fw 
filter parent 1: protocol ip pref 2 fw handle 0x28 classid 1:40 
filter parent 1: protocol ip pref 3 fw 
filter parent 1: protocol ip pref 3 fw handle 0x14 classid 1:20 
filter parent 1: protocol ip pref 6 fw 
filter parent 1: protocol ip pref 6 fw handle 0x1e classid 1:30

1:40 is my IPTV that should be around 12Mbps (HD channel) but now shares the BW with the wget running in class 1:20 that as you can see has lower prio.

Any idea? This is a real proiblem, because the available bandwidth through my and most of the ISPs is not always the same.

Rudy and nbd also use the output en forward chain.

nbd's scripts uses HFSC and your script is based on HTB. You can’t compare them!

UPLOAD:
If you allocate more bandwidth than available through you ISP, you are filling up your modem’s queue. High RTT (ICMP packets) is the result of filling up your modem’s queue.

DOWNLOAD:
You have to allocate a bandwidth of max 90% of your available bandwidth AND use good leaf qdisc (RED, SFQ,...) and don't allow burst traffic.  Your ISP queue handles burst traffic.

(Last edited by TomVH on 29 Jan 2012, 14:42)

Indeed the default openwrt qos-scripts package is using hfsc, but since both are intended doing the same thing I do not see an obstacle to compare them, however finding the most effective solution is my goal. If look in the first post there are 2 versions of rudy's scrpt - one that uses htb and one that uses hfsc.... since I have not used them, my comparrioson was between my script and the qos-scripts package that is much different than these ones.

I am using sfq for my leaf classes, the problem is that my bandwith is not always constant, anyway I do not want to limit myself to 90% of my Internet speed when 100% of it is available. I suppose you mean to limit the ceil rate for every class to 90% of my Download speed, right?
Is there any other solution?

(Last edited by dir2cas on 29 Jan 2012, 16:38)

much different? What's the difference?

HTB and HFSC are the same?
I’ve never worked with HTB but can you guarantee low delay for VoIP packets? With HFSC it’s possible.

right!

UPLOAD:
Set the max root class rate (ceil in htb and ul in HFSC) to the max linespeed minus the overhead.
Example:
I have a upload line speed of 640 kbps. If I use only applications which send small packets, the overhead can be 50% of your bandwidth. So I have to set the “ceil parameter” to 320 kbit.
If I use only applications which send large packets, the overhead is small.
If I have applications which send large AND small packets, I can use the TC stab option and set “ceil” parameter (ul in HFSC) to 620 kbit (maybe 640 kbit is also possible).

It’s IMPOSSIBLE to guarantee low delay and bandwidth allocations if the ceil parameter is not lower than the max linespeed (minus the overhead if you don’t use the tc stab option).

DOWNLOAD
If you will guarantee low delay and bandwidth, you have to set the “ceil”parameter much lower than the max linespeed.
At home I have a linespeed of 6140 kbit and I set the ul parameter (ceil in htb) of the root class to 5000kbit. 20 % lower than the max linespeed !!
Why:
It’s only possible to slow down TCP and(and some udp) traffic to drop packets AND/OR make large queues at your router. It’s only possible to fill the router’s queue if you set the max speed (much) lower than the max linespeed.

It’s IMPOSSIBLE to guarantee low delay and bandwidth allocations if the ceil parameter is not lower than the max linespeed (minus the overhead if you don’t use the tc stab option).

If your ISP delivers you a variable speed: read this

I think that you did not get it right! I did not say that htb and hfsc are the same, but that the general (not meaning the particular features of both of them) idea behind them is same - QoS.

Thanks for the link, I will read it and try to figure out if I will be able to implement it in my script. I wanted to finish it fully before posting any code here, so excuse me if I do not provide the full details in order to make the troubleshooting easier.

I don't understand why you use HTB. A couple of months ago, I read a lot of documentation. After reading this documentation, I was convinced to use HFSC instead of HTB. And you?

HFSC scheduling with linux
For complex traffic shaping scenarios, hierarchical algorithms are necessary. Current versions of Linux support the algorithms HTB and HFSC. While HTB basically rearranges token bucket filter (TBF) into a hierarchical structure, thereby retaining the principal characteristics of TBF, HFSC allows proportional distribution of bandwidth as well as control and allocation of latencies. This allows for better and more efficient use of a connection for situations in which both bandwidth intensive data services and interactive services share a single network link.

HIERARCHICAL FAIR SERVICE CURVE  History & Introduction
In short HFSC aims to:

1) guarantee precise bandwidth and delay allocation for all leaf classes (realtime criterion)
2) allocate excess bandwidth fairly as specified by class hierarchy (linkshare & upperlimit criterion)
3) minimize any discrepancy between the service curve and the actual amount of service provided during linksharing

The main "selling" point of HFSC is feature (1), which is achieved by using nonlinear service curves (more about what it actually is later). This is particularly useful in VoIP or games, where not only guarantee of consistent bandwidth is important, but initial delay of a data stream as well. Note that it matters only for leaf classes (where the actual queues are) - thus class hierarchy is ignored in realtime case.

Feature (2) is well, obvious - any algorithm featuring class hierarchy (such as HTB or CBQ) strives to achieve that. HFSC does that well, although you might end with unusual situations, if you define service curves carelessly - see section CORNER CASES for examples.

Feature (3) is mentioned due to the nature of the problem. There may be situations where it's either not possible to guarantee service of all curves at the same time, and/or it's impossible to do so fairly. Both will be explained later. Note that this is mainly related to interior (aka aggregate) classes, as the leafs are already handled by (1). Still - it's perfectly possible to create a leaf class w/o realtime service, and in such case - the caveats will naturally extend to leaf classes as well.

Please tell us in words the difference between your script and nbd's one / rudy's one. What are the different goals/objectives?
The goals of my script:

• use of 2 VoIP applications with a max delay of 10 ms upload and 1 ms download => HFSC

• high bandwidth allocation to HTTP and other "important" traffic => IP tables and HFSC

• use of P2P applications (µtorrent) if bandwidth is available => HFSC and RED

• max throughput => TC stab

result: see the first picture of my post "Qdisc hierarchy"
My linespeeds are stable, so I don't need the speed correction.

I think the nbd's script is good but:

• there is a lack of documentation. I will write some documentation if the leaf qdisc are configurable and the TC stab option is implemented.

• no overhead calculation

• leaf qdisc are not configurable

• no speed correction

• maybe wrong calculation of service curves.

Ok, If I have some time, I will participate but i think it's much much much more important to write some documentation of the nbd's script, implement the stab option and make it possible to configure the leaf qdiscs.

TomVH, thanks for sharing your experince, anyway, I've also read about HFSC (not so few as you think) and linux tc and packet scheduler machine. The main reason for using HTB is that its configuration is cleaner and simplier and because of its realtive cpu effectiveness, that is important when dealing with QoS on an embedded device.

Since I have pretty wide WAN bandwidth (24 Mbps is an average linespeed in my country, but it is enough for me) the main goal was to be able to download files with the maximum possible speed without eating the needed bandwidth for my IPTV (especially while watching HD channels) and of course - wanted to do it on my own, starting from scratch and learn how all this works as well. In fact my script is doing the job pretty good. I included several additional features in my design keeping it as highly customizable as possible. Well, as I wrote the only problem I faced till now is the link saturation varying the linespeed from my provider from time to time. Initially I thought that the prio parameter of tc (and htb in particular) would cope with that, but as you explained without setting the proper bandwidth limit to be under your linespeed provided by your ISP - it will not be possible.

I am now interested in tc stab option.

Since it aims to calculate L2 protocol overhead properly, how it aims to outsand variations of the available linespeed? I am ready to alter my architecture using tc stab and hfsc as classful queueing if I can achieve proper qos traffic handling without doing link cotrollers, etc, like suggested in the link to the Gargoyle thread that you gave above.

(Last edited by dir2cas on 30 Jan 2012, 17:06)

Where have you read that HTB is more “CPU effective”

I suppose that you have a xDSL line and not cable (coax)?

Here (in Belgium) there are different xDSL profiles. Example the 30/6 or the 50/6 “profile”. So the download sync speed of the line is 30000kbit (bit per seconds) and the upload sync speed is 6000kbit. These values shows how many packets could be send on the wire (inclusive PPP,… overhead). BUT this does not mean that ISP delivers you 30000kbit. Here the ISP limit the available bandwidth to 25000kbit download / 3000kbit upload. So you can’t use (30000kbit-25000kbit=)5000kbit. BUT the 5000kbit can be used by IPTV. I don’t know if I use the correct terms (sync speed,…)

Suppose a IPTV use a bandwidth of 6000bit.
0 IPTV’s active => 25000kbit available (for browsing, downloading,….)
1 IPTV active => 30000kbit-6000kbit=24000kbit available (for browsing, downloading,….)
2 IPTV’s active => 30000kbit – 12000kbit = 18000kbit available (for browsing, downloading,….)

Maybe is this causes the variable speed?

Here the ISP delivers a modem/router and use Vlans to separate IPTV / VOIP and Internet. Below a suggestion to solve this problem:

I don’t have a fast VDSL link and no IPTV because VDSL is not available here. I have only a slow adsl link (6140kbit/640kbit). So I have no experience with IPTV and vlans,.... It's just a suggestion.

TC-stab only modify the packet size calculation to take overhead into account. It do not solve the variable linespeed problem.

“speed controlling” like Gargoyle is really necessary if:
-your xdsl speed isn’t constant. But I think xdsl (ADSL / VDSL) sync speed are always constant? Maybe you can read something about VDSL to find out the right answer. I only know that VDSL don’t use ATM encapsulation but Ethernet frames.
- your ISP limit your speed and this limitations isn’t constant.

Why I use HFSC? (below the explanations for upload)
I think with HTB you can’t control the real-time speed. If a class start sending, the first packet(s) (=burst) are sent at maximum speed. So the max available bandwidth is exceeded. If you don’t use burst in HTB, it’s impossible to guarantee low delays:
Example: Voip packets are 400byte. Assume I have a upload speed of 1Mbit. => it takes 400byte/1Mbit=3,2 ms to send a packet. The VoIP app sends packets at 100 kbit = 31 packets per second. In order to meet het guaranteed 100kbit, the qdisc must send a packet every (1s/31packets=)32 ms. The max delay of HTB is 32 ms.
In HFSC you can reduce the delay to 3,2 ms.

Have you already test your script with ICMP packets?.
I have a line speed of 640kbit
If my line is empty, the RTT of ICMP packets to 8.8.8.8 (google dns) is 22 ms.
ICMP packets are 84byte long. The first slope of the prio class is 300kbit. Assume there is a packet overhead of 50 %. So the packet size = 168byte . => every (168*8/300=)4,48 ms HFSC dequeues a packet of the prio class.
If there is already a packet of 1500byte in the modem's queue, it takes (1500*8+overhead)/640kbit= 20 ms to send a packet.
IF HFSC does his job, the total RTT may never exceeds (22ms + 20ms +4,48ms = ) 46,48 ms.
If I saturate the uplink, the delay of ICMP packet never exceed 45 ms. So IT WORKS

(Last edited by TomVH on 4 Feb 2012, 10:48)

I'm curious, why use the OUTPUT and FORWARD chains in the mangle table? I've got my own little QoS script that I use and I hook everything into the POSTROUTING chain. This captures both forwarding traffic and also router sourced traffic. This is the relevant part of my script:

DEV=eth1
iptables -t mangle -N adam_qos_$DEV &> /dev/null
iptables -t mangle -F adam_qos_$DEV &> /dev/null
iptables -t mangle -I POSTROUTING -o $DEV -j adam_qos_$DEV

Is there some reason NOT to do this? Seems to be the next to last step in the iptables order of operation for an outbound packet, right before POSTROUTING of the nat table.

RIGHT!
In case you have correctly optimized packet matching rules and right placed CONNMARK targets you may achieve smooth traffic classification even based on source IP address (from LAN - matched in egress, and source addresses coming from WAN side - matched in ingress). This all is possible because POSTROUTING chain first passes through the mangle table and after that is parsed by the nat table. This lets us to match packets generated from LAN's IP addresses (hosts) in the  POSTROUTING mangle table and classify them correctly in both directions if using properly set CONNMARK.

In fact I optimized my script to do exactly that - another 2-3% of CPU gain for other tasks.

However the problem when your linespeed is less than the defined bandwidth limit is still present. TomVH explained why in the above posts.

maybe there is a reason why the OpenWrt QoS developer(s) uses the output and forward chain but in my script I can use the postrouting chain instead of the output and forward chain.