1 (edited by rudy 2006-11-03 16:14:45)

Topic: A qos script that really works!


======================================================================
== Edit: The latest version is 1.05. The description can be found here:
== http://forum.openwrt.org/viewtopic.php?pid=36509#p36509
== The restated goals are the following:
==  -allow heavy p2p traffic without affecting web browsing, games or other interactive apps
==  -enable VoIP calls not impacted by any other traffic
==  -scalable configuration that adapts to a wide range of line speeds
==  -easy configuration
======================================================================

Finally, I managed to come up with a qos script that fullfills my simple requirement:

  Comfortably surf the web while running several p2p apps at unthrottled speeds

Before writing my own, I tried most every script known to Google. They all failed for several reasons:

1. p2p packet identification was poor, especially when using non-standard port. This turned out to be the case for layer7 as well is ipp2p.
2. ingress throttling was either absent or non-functional
3. configuration of the various qdisc was less than optimal

The fixes for these shortcomings in the final script are as follows:

1. Packet identification is done based on destination ports for outgoing traffic only. The classification is saved on the connection, so incoming packets are identified as well. Besides being very simple, this works without a hitch, since all p2p traffic is to high ports.

2. Included a htb qdisc to the outgoing traffic of the LAN interface. This effectively acts as an ingress discipline for the WAN port.

3. Added a hfsc discipline with balanced rates (they all add up to the parent rate) to the WAN interface.  The htb discipline on the LAN port has a long time constant of 1000ms to allow for the long round trip time of internet traffic.

The results are amazing, as evidenced by this email. I am typing up in the web interface of the forum, while running 3 uTorrent downloads at unlimited rates!

Go ahead and give it a try. Let me know if it works for you!

Be sure to set the upload and download rate (slightly) lower than your ADSL or Cable rates to keep modem and ISP queues from filling up, killing interactivity. My measured download and upload rates are 900kb/s and 300kb/s respectively. Setting the speeds in the script to 800kb/s and 280kb/s did the trick.

Rudy.

#!/bin/sh

DEBUG=0

# To enable logging (requires iptables-mod-extra package)
[ $DEBUG -eq 1 ] && insmod ipt_LOG >&- 2>&-

#######################################################
DOWNLOAD=800
UPLOAD=280
D=100
BURST=1000

TCP_BULK="1024:"
UDP_BULK="1024:"

TCP_PRIO="22 23 53 80 443"
UDP_PRIO="53"

#BULK_PROTOS="edonkey bittorrent"
#######################################################

WAN=$(nvram get wan_ifname)
LAN=$(nvram get lan_ifname)

U_M1_PRIO=$(($UPLOAD*90/100))
U_M1_NORM=$(($UPLOAD*10/100))
U_M1_BULK=$(($UPLOAD* 0/100))

U_M2_PRIO=$(($UPLOAD*50/100))
U_M2_NORM=$(($UPLOAD*30/100))
U_M2_BULK=$(($UPLOAD*20/100))

D_BURST=$(($BURST*$DOWNLOAD/8))

insmod cls_fw >&- 2>&-
insmod sch_hfsc >&- 2>&-
insmod sch_htb >&- 2>&-
insmod ipt_CONNMARK >&- 2>&-
insmod ipt_length >&- 2>&-
insmod ipt_limit >&- 2>&-
insmod ipt_tos >&- 2>&-
#insmod sch_ingress >&- 2>&-
#insmod ipt_layer7 >&- 2>&-
#insmod ipt_ipp2p >&- 2>&-
#insmod ipt_multiport >&- 2>&-
#insmod cls_u32 >&- 2>&-

iptables -t mangle -F
iptables -t mangle -X

tc qdisc del dev $WAN root >&- 2>&-
tc qdisc add dev $WAN root handle 1: hfsc default 30
tc class add dev $WAN parent 1: classid 1:1 hfsc sc rate ${UPLOAD}kbit ul rate ${UPLOAD}kbit
tc class add dev $WAN parent 1:1 classid 1:10 hfsc sc m1 ${U_M1_PRIO}kbit d ${D}ms m2 ${U_M2_PRIO}kbit ul rate ${UPLOAD}kbit
tc class add dev $WAN parent 1:1 classid 1:20 hfsc sc m1 ${U_M1_NORM}kbit d ${D}ms m2 ${U_M2_NORM}kbit ul rate ${UPLOAD}kbit
tc class add dev $WAN parent 1:1 classid 1:30 hfsc sc m1 ${U_M1_BULK}kbit d ${D}ms m2 ${U_M2_BULK}kbit ul rate ${UPLOAD}kbit
tc filter add dev $WAN parent 1: prio 1 protocol ip handle 1 fw flowid 1:10
tc filter add dev $WAN parent 1: prio 2 protocol ip handle 2 fw flowid 1:20
tc filter add dev $WAN parent 1: prio 3 protocol ip handle 3 fw flowid 1:30

#tc qdisc del dev $WAN ingress >&- 2>&-
#tc qdisc add dev $WAN handle ffff: ingress
#tc filter add dev $WAN parent ffff: protocol ip prio 50 handle 3 fw police rate $(($DOWNLOAD/2))kbit burst $D_BURST drop flowid :1
#tc filter add dev $WAN parent ffff: protocol ip prio 50 u32 match ip src 0.0.0.0/0 police rate $(($DOWNLOAD))kbit burst $D_BURST drop flowid :1

tc qdisc del dev $LAN root >&- 2>&-
# htb qdisc without default: all unmarked (mark 0) packages pass unlimited
tc qdisc add dev $LAN root handle 1: htb
tc class add dev $LAN parent 1: classid 1:1 htb rate ${DOWNLOAD}kbit ceil ${DOWNLOAD}kbit burst $D_BURST cburst $D_BURST
tc class add dev $LAN parent 1:1 classid 1:10 htb rate $(($DOWNLOAD*8/10))kbit ceil ${DOWNLOAD}kbit burst $D_BURST cburst $D_BURST prio 0
tc class add dev $LAN parent 1:1 classid 1:20 htb rate $(($DOWNLOAD*2/10))kbit ceil $(($DOWNLOAD/2))kbit burst $D_BURST cburst $D_BURST prio 1
tc filter add dev $LAN parent 1: prio 1 protocol ip handle 1 fw flowid 1:10
tc filter add dev $LAN parent 1: prio 2 protocol ip handle 2 fw flowid 1:10
tc filter add dev $LAN parent 1: prio 3 protocol ip handle 3 fw flowid 1:20

iptables -t mangle -N mark_chain
iptables -t mangle -N ingress_chain

iptables -t mangle -A POSTROUTING -o $WAN -j mark_chain
iptables -t mangle -A PREROUTING -i $WAN -j ingress_chain

###################################### INGRESS CHAIN #################################################
# Restore any saved connection mark
iptables -t mangle -A ingress_chain -j CONNMARK --restore-mark

# Default is normal priority (to make sure every packet on wan interface gets marked)
iptables -t mangle -A ingress_chain -m mark --mark 0 -j MARK --set-mark 2

# Mark *any* p2p package (first package in connection only)
#iptables -t mangle -A ingress_chain -m mark --mark 0 -m ipp2p --ipp2p -j MARK --set-mark 1

# Mark bulk packets according to Layer 7 match. Works for first package only!
#for PROTO in $BULK_PROTOS; do
#  iptables -t mangle -A ingress_chain -m mark --mark 0 -m layer7 --l7proto $PROTO -j MARK --set-mark 1
#done

# Save mark onto connection
#iptables -t mangle -A ingress_chain -j CONNMARK --save-mark
######################################################################################################
    
######################################## MARK CHAIN ##################################################
# Restore any saved connection mark
iptables -t mangle -A mark_chain -j CONNMARK --restore-mark

# Mark prio packets based on port numbers and protocol
for PORT in $UDP_PRIO; do
  iptables -t mangle -A mark_chain -m mark --mark 0 -p udp --dport $PORT -j MARK --set-mark 1
done
for PORT in $TCP_PRIO; do
  iptables -t mangle -A mark_chain -m mark --mark 0 -p tcp --dport $PORT -j MARK --set-mark 1
done

# Mark bulk packets based on port numbers and protocol
for PORT in $UDP_BULK; do
  iptables -t mangle -A mark_chain -m mark --mark 0 -p udp --dport $PORT -j MARK --set-mark 3
done
for PORT in $TCP_BULK; do
  iptables -t mangle -A mark_chain -m mark --mark 0 -p tcp --dport $PORT -j MARK --set-mark 3
done

# Save mark onto connection
iptables -t mangle -A mark_chain -j CONNMARK --save-mark

# Make sure ACK packets get priority
iptables -t mangle -A mark_chain -p tcp -m length --length :128 --tcp-flags SYN,RST,ACK ACK -j MARK --set-mark 1

# Default is normal priority
iptables -t mangle -A mark_chain -m mark --mark 0 -j MARK --set-mark 2
######################################################################################################

[ $DEBUG -eq 1 ] && iptables -t mangle -A mark_chain -m mark --mark 0 -j LOG --log-prefix mark_0::
[ $DEBUG -eq 1 ] && iptables -t mangle -A mark_chain -m mark --mark 0 -j ACCEPT
[ $DEBUG -eq 1 ] && iptables -t mangle -A mark_chain -m mark --mark 1 -j LOG --log-prefix mark_1::
[ $DEBUG -eq 1 ] && iptables -t mangle -A mark_chain -m mark --mark 1 -j ACCEPT
[ $DEBUG -eq 1 ] && iptables -t mangle -A mark_chain -m mark --mark 2 -j LOG --log-prefix mark_2::
[ $DEBUG -eq 1 ] && iptables -t mangle -A mark_chain -m mark --mark 2 -j ACCEPT
[ $DEBUG -eq 1 ] && iptables -t mangle -A mark_chain -m mark --mark 3 -j LOG --log-prefix mark_3::
[ $DEBUG -eq 1 ] && iptables -t mangle -A mark_chain -m mark --mark 3 -j ACCEPT
[ $DEBUG -eq 1 ] && iptables -t mangle -A mark_chain -j LOG --log-prefix mark_other::

[ $DEBUG -eq 1 ] && iptables -t mangle -A ingress_chain -m mark --mark 0 -j LOG --log-prefix ingress_0::
[ $DEBUG -eq 1 ] && iptables -t mangle -A ingress_chain -m mark --mark 0 -j ACCEPT
[ $DEBUG -eq 1 ] && iptables -t mangle -A ingress_chain -m mark --mark 1 -j LOG --log-prefix ingress_1::
[ $DEBUG -eq 1 ] && iptables -t mangle -A ingress_chain -m mark --mark 1 -j ACCEPT
[ $DEBUG -eq 1 ] && iptables -t mangle -A ingress_chain -m mark --mark 2 -j LOG --log-prefix ingress_2::
[ $DEBUG -eq 1 ] && iptables -t mangle -A ingress_chain -m mark --mark 2 -j ACCEPT
[ $DEBUG -eq 1 ] && iptables -t mangle -A ingress_chain -m mark --mark 3 -j LOG --log-prefix ingress_3::
[ $DEBUG -eq 1 ] && iptables -t mangle -A ingress_chain -m mark --mark 3 -j ACCEPT
[ $DEBUG -eq 1 ] && iptables -t mangle -A ingress_chain -j LOG --log-prefix ingress_other::

Re: A qos script that really works!

Hi,
i just tried out your script with the following errors

root@OpenWrt:~# /etc/fireqos.sh
iptables v1.3.3: Unknown arg `--restore-mark'
Try `iptables -h' or 'iptables --help' for more information.
iptables v1.3.3: Unknown arg `--restore-mark'
Try `iptables -h' or 'iptables --help' for more information.
iptables v1.3.3: Unknown arg `--save-mark'
Try `iptables -h' or 'iptables --help' for more information.
iptables v1.3.3: Couldn't load match `length':File not found

Try `iptables -h' or 'iptables --help' for more information.


im running a wrt54g

i first tried your script with rc3 with a few more errors, i upgraded to rc4, atm its a basic install with a few packages that would not affect firewalling. have install the tc package.

dont think its working, will do some further tests tho

Re: A qos script that really works!

Hi,

Looks like you're missing some kernel modules. I think you will need these to make the script work:

kmod-sched
kmod-ipt-conntrack
iptables-mod-conntrack
kmod-ipt-ipopt
iptables-mod-ipopt
kmod-ipt-extra
iptables-mod-extra

(Install each module with with "ipkg install <package>". Be sure to run "ipkg update" before doing the installs.)

To get more verbose output and error messages, you can take out the ">&- 2>&-" behind each of the insmod commands. They are there to avoid messages when the scripts tries to load modules that are already in memory, but they also hide any more serious error message. I noticed the ipt_tos module is not used in the script anymore, and can be deleted/commented out.

Re: A qos script that really works!

Hi Rudy,
thanks for the reply. After installing those modules script now runs ok but im having speed problems now.
Sorry im not very good with iptables so bear with me smile

In your original post you talked about setting the speeds in kb/s im guessing you are talking kilobits per second?

here is my current settings

DOWNLOAD=450
UPLOAD=100
D=100
BURST=1000

TCP_BULK="1024:"
UDP_BULK="1024:"

TCP_PRIO="22 23 53 80 443"
UDP_PRIO="53"


im on a 512/128 ADSL connection. with the settings like that while grabbing a file off my isp's ftp will sit around 25-30KB/s browsing is very fast. with the script off maxes out the adsl connection but basically cannot browse.

I tried those varibles in KB/s i think it was 45 down and 10 up, everything ran very slowly.
What am i doing wrong?

cheers

Re: A qos script that really works!

Hi,

Sounds like at least the basic QoS functionality is up now. If I understand you correctly, the ftp download runs at about half the speed with the script as compared to without it?

Can you check if the same applies to http downloads? Maybe check on one of those ADSL speed check sites.

One other test is to see if the ftp traffic gets inadvertedly marked as bulk (lowest priority). In the script, you can change the line

tc class add dev $LAN parent 1:1 classid 1:20 htb rate $(($DOWNLOAD*2/10))kbit ceil $(($DOWNLOAD/2))kbit burst $D_BURST cburst $D_BURST prio 1

to

tc class add dev $LAN parent 1:1 classid 1:20 htb rate $(($DOWNLOAD*2/10))kbit ceil $(($DOWNLOAD))kbit burst $D_BURST cburst $D_BURST prio 1

See if this makes any difference.

The script limits download rates for bulk traffic to half the available bandwidth. This assures there is always download bandwidth available for higher priority traffic. Although it should theoretically be possible to download bulk traffic at the full rate, until higher priority traffic arrives, in practice the long round trip delays of the internet makes that the ingress line sharing is to slow to assure a fast response to high priority packets.

Anyhow, since you didn't mark ftp ports as bulk, the ftp connection should not be speed limited, and you shouldn't see the slow down from the script. I'd be curious to hear your feedback on the tests.

Re: A qos script that really works!

O.k., I think I found the problem. You're probably using a web browser for the ftp download, making a passive FTP connection. Turns out the script had a particular problem with passive FTP. The way it was set up, it didn't correctly save the priority onto the connection, then deciding it was a bulk connection, based on the high port numbers being used. I changed a couple of lines around, and it should work now (it does for me). Please check it out and let me know if it works for you too.

If you were to decide to demote ftp traffic to bulk status again, along with p2p, just add port 21 to the TCP_BULK variable (space separated list).

This is the updated script, commented out commands of the previous version deleted:

#!/bin/sh

DEBUG=0

# To enable logging (requires iptables-mod-extra package)
[ $DEBUG -eq 1 ] && insmod ipt_LOG >&- 2>&-

#######################################################
DOWNLOAD=800
UPLOAD=280

TCP_BULK="1024:"
UDP_BULK="1024:"

TCP_PRIO="22 23 53 80 443"
UDP_PRIO="53"
#######################################################

D=100
BURST=1000

WAN=$(nvram get wan_ifname)
LAN=$(nvram get lan_ifname)

U_M1_PRIO=$(($UPLOAD*90/100))
U_M1_NORM=$(($UPLOAD*10/100))
U_M1_BULK=$(($UPLOAD* 0/100))

U_M2_PRIO=$(($UPLOAD*50/100))
U_M2_NORM=$(($UPLOAD*30/100))
U_M2_BULK=$(($UPLOAD*20/100))

D_BURST=$(($BURST*$DOWNLOAD/8))

# The following packages are required for the modules:
# kmod-sched
# kmod-ipt-conntrack
# iptables-mod-conntrack
# kmod-ipt-ipopt
# iptables-mod-ipopt
# kmod-ipt-extra
# iptables-mod-extra

insmod cls_fw >&- 2>&-
insmod sch_hfsc >&- 2>&-
insmod sch_htb >&- 2>&-
insmod ipt_CONNMARK >&- 2>&-
insmod ipt_length >&- 2>&-
insmod ipt_limit >&- 2>&-

iptables -t mangle -F
iptables -t mangle -X

tc qdisc del dev $WAN root >&- 2>&-
tc qdisc add dev $WAN root handle 1: hfsc default 30
tc class add dev $WAN parent 1: classid 1:1 hfsc sc rate ${UPLOAD}kbit ul rate ${UPLOAD}kbit
tc class add dev $WAN parent 1:1 classid 1:10 hfsc sc m1 ${U_M1_PRIO}kbit d ${D}ms m2 ${U_M2_PRIO}kbit ul rate ${UPLOAD}kbit
tc class add dev $WAN parent 1:1 classid 1:20 hfsc sc m1 ${U_M1_NORM}kbit d ${D}ms m2 ${U_M2_NORM}kbit ul rate ${UPLOAD}kbit
tc class add dev $WAN parent 1:1 classid 1:30 hfsc sc m1 ${U_M1_BULK}kbit d ${D}ms m2 ${U_M2_BULK}kbit ul rate ${UPLOAD}kbit
tc filter add dev $WAN parent 1: prio 1 protocol ip handle 1 fw flowid 1:10
tc filter add dev $WAN parent 1: prio 2 protocol ip handle 2 fw flowid 1:20
tc filter add dev $WAN parent 1: prio 3 protocol ip handle 3 fw flowid 1:30

tc qdisc del dev $LAN root >&- 2>&-
# htb qdisc without default: all unmarked (mark 0) packages pass unlimited
tc qdisc add dev $LAN root handle 1: htb
tc class add dev $LAN parent 1: classid 1:1 htb rate ${DOWNLOAD}kbit ceil ${DOWNLOAD}kbit burst $D_BURST cburst $D_BURST
tc class add dev $LAN parent 1:1 classid 1:10 htb rate $(($DOWNLOAD*8/10))kbit ceil ${DOWNLOAD}kbit burst $D_BURST cburst $D_BURST prio 0
tc class add dev $LAN parent 1:1 classid 1:20 htb rate $(($DOWNLOAD*2/10))kbit ceil $(($DOWNLOAD/2))kbit burst $D_BURST cburst $D_BURST prio 1
tc filter add dev $LAN parent 1: prio 1 protocol ip handle 1 fw flowid 1:10
tc filter add dev $LAN parent 1: prio 2 protocol ip handle 2 fw flowid 1:10
tc filter add dev $LAN parent 1: prio 3 protocol ip handle 3 fw flowid 1:20

iptables -t mangle -N mark_chain
iptables -t mangle -N ingress_chain

iptables -t mangle -A POSTROUTING -o $WAN -j mark_chain
iptables -t mangle -A PREROUTING -i $WAN -j ingress_chain

###################################### INGRESS CHAIN #################################################
# Restore any saved connection mark
iptables -t mangle -A ingress_chain -j CONNMARK --restore-mark

# Default is normal priority (to make sure every packet on wan interface gets marked)
iptables -t mangle -A ingress_chain -m mark --mark 0 -j MARK --set-mark 2
######################################################################################################
    
######################################## MARK CHAIN ##################################################
# Restore any saved connection mark
iptables -t mangle -A mark_chain -j CONNMARK --restore-mark

# Mark prio packets based on port numbers and protocol
for PORT in $UDP_PRIO; do
  iptables -t mangle -A mark_chain -m mark --mark 0 -p udp --dport $PORT -j MARK --set-mark 1
done
for PORT in $TCP_PRIO; do
  iptables -t mangle -A mark_chain -m mark --mark 0 -p tcp --dport $PORT -j MARK --set-mark 1
done

# Mark bulk packets based on port numbers and protocol
for PORT in $UDP_BULK; do
  iptables -t mangle -A mark_chain -m mark --mark 0 -p udp --dport $PORT -j MARK --set-mark 3
done
for PORT in $TCP_BULK; do
  iptables -t mangle -A mark_chain -m mark --mark 0 -p tcp --dport $PORT -j MARK --set-mark 3
done

# Default is normal priority
iptables -t mangle -A mark_chain -m mark --mark 0 -j MARK --set-mark 2

# Save mark onto connection
iptables -t mangle -A mark_chain -j CONNMARK --save-mark

# Make sure ACK packets get priority
iptables -t mangle -A mark_chain -p tcp -m length --length :128 --tcp-flags SYN,RST,ACK ACK -j MARK --set-mark 1
######################################################################################################

[ $DEBUG -eq 1 ] && iptables -t mangle -A mark_chain -m mark --mark 0 -j LOG --log-prefix mark_0::
[ $DEBUG -eq 1 ] && iptables -t mangle -A mark_chain -m mark --mark 0 -j ACCEPT
[ $DEBUG -eq 1 ] && iptables -t mangle -A mark_chain -m mark --mark 1 -j LOG --log-prefix mark_1::
[ $DEBUG -eq 1 ] && iptables -t mangle -A mark_chain -m mark --mark 1 -j ACCEPT
[ $DEBUG -eq 1 ] && iptables -t mangle -A mark_chain -m mark --mark 2 -j LOG --log-prefix mark_2::
[ $DEBUG -eq 1 ] && iptables -t mangle -A mark_chain -m mark --mark 2 -j ACCEPT
[ $DEBUG -eq 1 ] && iptables -t mangle -A mark_chain -m mark --mark 3 -j LOG --log-prefix mark_3::
[ $DEBUG -eq 1 ] && iptables -t mangle -A mark_chain -m mark --mark 3 -j ACCEPT
[ $DEBUG -eq 1 ] && iptables -t mangle -A mark_chain -j LOG --log-prefix mark_other::

[ $DEBUG -eq 1 ] && iptables -t mangle -A ingress_chain -m mark --mark 0 -j LOG --log-prefix ingress_0::
[ $DEBUG -eq 1 ] && iptables -t mangle -A ingress_chain -m mark --mark 0 -j ACCEPT
[ $DEBUG -eq 1 ] && iptables -t mangle -A ingress_chain -m mark --mark 1 -j LOG --log-prefix ingress_1::
[ $DEBUG -eq 1 ] && iptables -t mangle -A ingress_chain -m mark --mark 1 -j ACCEPT
[ $DEBUG -eq 1 ] && iptables -t mangle -A ingress_chain -m mark --mark 2 -j LOG --log-prefix ingress_2::
[ $DEBUG -eq 1 ] && iptables -t mangle -A ingress_chain -m mark --mark 2 -j ACCEPT
[ $DEBUG -eq 1 ] && iptables -t mangle -A ingress_chain -m mark --mark 3 -j LOG --log-prefix ingress_3::
[ $DEBUG -eq 1 ] && iptables -t mangle -A ingress_chain -m mark --mark 3 -j ACCEPT
[ $DEBUG -eq 1 ] && iptables -t mangle -A ingress_chain -j LOG --log-prefix ingress_other::

Re: A qos script that really works!

Hi Rudy,

have you got some good links to HFSC documentation in english?
I'm having troubles in googling it!

Regards,

Wallace

Re: A qos script that really works!

What does mean that values :
D=100
BURST=1000

TCP_BULK="1024:"
UDP_BULK="1024:

Thanks for reply smile

Best regards
http://openwrt.pl - Polish OpenWrt Wiki
http://digi-led.pl - Electronic Parts

Re: A qos script that really works!

Some hfsc pointers:

http://www.cs.cmu.edu/~hzhang/HFSC/main.html (site appears down at time of writing)
http://www.cs.cmu.edu/~hzhang/papers/SIGCOM97.pdf
http://wsched.sourceforge.net/
http://wsched.sourceforge.net/thesis.pdf
http://trash.net/~kaber/hfsc/README

You can also try:
tc qdisc add hfsc help
on the OpenWRT command line

Re: A qos script that really works!

The values have the following meaning:

D=100
The hfsc algorithm for the outgoing traffic sets two different data rates, one for short bursts (m1), and one for sustained data (m2). The time D (in ms) is the time when the algorithm switches from m1 to m2.

BURST=1000
This is the size of the HTB bucket for incoming traffic in ms. The script calculates the number of bytes for the buffer based on the max download speed. Basically, this sets the burst window for the ingress shaping. On average, the HTB algorithm will make sure it satisfies the rate specified on the command line, but allows for temporary data bursts. BURST sets the maximum burst time (1s in the script). A long burst time will average the datarate over a longer time, allowing bursty internet traffic to get through unharmed.

TCP_BULK="1024:"
UDP_BULK="1024:"
These variables are space separated lists of ports to which traffic gets the bulk or lowest priority. The script works with three different priorities: prio, norm and bulk (mark 1, 2 and 3). All data is considered normal priority, unless the destination ports of the outgoing traffic are listed in one of the four variables TCP_PRIO, UDP_PRIO, TCP_BULK and UDB_BULK. The syntax of the variables is that of space separed entries, each of which represents a standard iptables port match. The value 1024: is a port range and simply means 1024 and above. A value of 1024:2000 would mean 1024 thru 2000 and :1024 matches all ports below 1024. Adding the standard ftp port 21 to the bulk ports requires a line like:

TCP_BULK="1024: 21"

Hope this helps,
Rudy.

Re: A qos script that really works!

Hi,

@rudy, why you use HTB for download and HFSC for upload? why not HFSC for both?

Thanks!

Re: A qos script that really works!

O.k., short question with a long answer. Please bear with me.

The reason for the different queuing disciplines for up and download is that the nature of the traffic for up and down is very different:

1. down (ingress) traffic is much "burstier" than up traffic, due to the shared nature of the internet.

2. down traffic has much longer round trip times.

3. the bottle neck for the down traffic, which is the WAN connection, is in fact in front of the place where traffic is being shaped, whereas for up (egress) traffic it comes behind the shaping.

As a result, the time constants for shaping the speed of the up and down connection at the router level need to be different.

Remember that speed control of a connection can only take place at the sending application level. What happens is that when traffic gets throtteled somewhere in the line by dropping packets, the connection hand shake gets broken up, as fewer and fewer packets are actually acknowledged by the receiveing end. The sending app recognizes that the pipe is clogging up and backs off the data rate until the packets start coming through again. In the mean time, the application will need to retransmit data for all the missing packets.

For the upload link, this speed control handshake loop is quite fast and able to control the speed (almost) instantly. Also, the extra bandwidth consumed by the retransmitted packets happens on the LAN side, which is typically not the speed bottleneck.

The download link, with its much burstier traffic, is a different story. Hard limiting (or policing) the instantaneous speed will cut into the peaks of the data rate, while troughs can be quite deep.  On average, this will lower the average speed of the connection. Furthermore, all retransmitted packets eat up bandwidth from the WAN line, which is typically our bottleneck to begin with. Imagine half the packets requiring retransmission, this will effectively half the maximum data rate. This is aggrevated by the fact that the sending app will respond slowly to missing ack packets, because of the long round trip delays, and continue sending data long after the router starts dropping packets, eating up even more bandwidth with retransmissions.

The solutions is to control the average data rate over some period of time. This is effectively what the burst parameter of the htb discipline does. Instantaneous overruns of the data rate are allowed, as long as the average speed remains constant. Setting the bucket length to 1 second assures that the peaks and troughs of the internet traffic get evened out.

The hfsc discpline is intended for egress shaping only, and acts on instantaneous data rates. It doesn't allow for averaging of the data rate. In practice, the maximum download speed will go down by a factor 3 or more if the script were to use hfsc instead of htb for the ingress shaping.

Re: A qos script that really works!

How appropriate would this script be for a setup with two LAN workstations mostly playing MMPORG type games with maybe a few downloads going on every now and then, and general web browsing during the game?

Re: A qos script that really works!

jeejoo wrote:

How appropriate would this script be for a setup with two LAN workstations mostly playing MMPORG type games with maybe a few downloads going on every now and then, and general web browsing during the game?

What would be your expectations in terms of priorities? Is it critical the game is real-time, whereas the web browsing can wait, or do you want the game to take second seat to web browsing, making sure the web browsing is unaffected by game play? Also, what type of downloads do you expect, ftp, p2p or any other?

Re: A qos script that really works!

rudy wrote:
jeejoo wrote:

How appropriate would this script be for a setup with two LAN workstations mostly playing MMPORG type games with maybe a few downloads going on every now and then, and general web browsing during the game?

What would be your expectations in terms of priorities? Is it critical the game is real-time, whereas the web browsing can wait, or do you want the game to take second seat to web browsing, making sure the web browsing is unaffected by game play? Also, what type of downloads do you expect, ftp, p2p or any other?

I would want the games to have highest priority.  Web browsing and downloading can be lower priorities.  The downloading would be http and some bittorrents.  Thanks for your consideration in this matter.  I am studying all these QoS/L7/Traffic Shaping concepts and this will help me learn a bit more I think.

Re: A qos script that really works!

jeejoo wrote:

I would want the games to have highest priority.  Web browsing and downloading can be lower priorities.  The downloading would be http and some bittorrents.  Thanks for your consideration in this matter.  I am studying all these QoS/L7/Traffic Shaping concepts and this will help me learn a bit more I think.

I have had bad experience with the L7 packet filters. It seemed to have a hard time identifying peer-to-peer traffic. Matching based on standard ports gives more reliable results and is generally simpler, especially combined with the Netfilter CONNMARK module. What are the ports used for your MMORPG games?

Re: A qos script that really works!

rudy wrote:
jeejoo wrote:

I would want the games to have highest priority.  Web browsing and downloading can be lower priorities.  The downloading would be http and some bittorrents.  Thanks for your consideration in this matter.  I am studying all these QoS/L7/Traffic Shaping concepts and this will help me learn a bit more I think.

I have had bad experience with the L7 packet filters. It seemed to have a hard time identifying peer-to-peer traffic. Matching based on standard ports gives more reliable results and is generally simpler, especially combined with the Netfilter CONNMARK module. What are the ports used for your MMORPG games?

Ah that is part of is making the learning curve on this topic so steep.  There are many different approaches it seems.  L7, tc, HFSC, etc etc.  Some overlap and interact and do different specific things that all add up to 'QoS' from what I understand so far, which unfortunately isn't enough yet.  I wonder if there is an O'Reilly book on this yet? wink

Anyway, the game is World of Warcraft and it uses port 3724.  Thanks again.

Re: A qos script that really works!

jeejoo wrote:

Ah that is part of is making the learning curve on this topic so steep.  There are many different approaches it seems.  L7, tc, HFSC, etc etc.  Some overlap and interact and do different specific things that all add up to 'QoS' from what I understand so far, which unfortunately isn't enough yet.  I wonder if there is an O'Reilly book on this yet? wink

Anyway, the game is World of Warcraft and it uses port 3724.  Thanks again.

You are right, QoS under Linux can take some perseverance. The good news is that the Linux kernel allows for pretty nifty traffic shaping, and the smart people from OpenWRT produced a wonderful port for the Broadcom routers that makes it easy to experiment.
Getting back to your question, I think the script will work for you, pretty much as is. The only thing to add is the WoW port to the TCP_PRIO variable. The line would read like this:

TCP_PRIO="22 23 53 80 443 3724"

Let me know if it works.

Re: A qos script that really works!

rudy wrote:

Some hfsc pointers:
[...]
http://wsched.sourceforge.net/
[...]
tc qdisc add hfsc help
on the OpenWRT command line

Thank you for the links!

I noticed that 'tc qdisc add hfsc help' doesn't give the same output as in wsched examples page.
Wireless extensions disabled?

I found an ip-wsched ipkg on Florian Farinelli's repository (openwrt.alphacore.net) but it's just iproute2 binary... are there somewhere kernels patched to use wsched wireless shaping tools?

Regards

Wallace

Re: A qos script that really works!

I haven't seen packages to implement wsched on OpenWRT, but I didn't really look for it. I gave the pointer, because I found the documentation useful to understand the 'standard' hfsc algorithm. As you have found, documentation on this topic is very scarce!

What is it you want to accomplish with wsched?

Re: A qos script that really works!

rudy wrote:

What is it you want to accomplish with wsched?

1) "variable rate"

It could be usefull, as an example, to set rate based on olsr ETX... and/or rssi/noise ratio...
Somehow to reduce waste of bandwidth related to the fixed rate (ceil/max/cap).

2) "resource-consumption aware scheduling"

Example: To be sure that in case of voip traffic peaks, all the rest is slowed down.

ciao

Wallace

Re: A qos script that really works!

Good work! I just implemented this on a box. Speeded up my ssh connection in from outside nicely.

Re: A qos script that really works!

vincentfox wrote:

Good work! I just implemented this on a box. Speeded up my ssh connection in from outside nicely.

Thanks for the feedback!

Re: A qos script that really works!

Wallace78 wrote:

1) "variable rate"

It could be usefull, as an example, to set rate based on olsr ETX... and/or rssi/noise ratio...
Somehow to reduce waste of bandwidth related to the fixed rate (ceil/max/cap).

2) "resource-consumption aware scheduling"

Example: To be sure that in case of voip traffic peaks, all the rest is slowed down.

ciao

Wallace

Beyond my basic needs... but exciting stuff.

Re: A qos script that really works!

vincentfox wrote:

Good work! I just implemented this on a box. Speeded up my ssh connection in from outside nicely.

Anyone else that has experience with the script? Did it work for you? I am especially interested to hear how it holds up in a VOIP environment. The script should be able to guarantuee solid VOIP performance by adding the SIP and RTP ports to the PRIO variables. Let me know what you've found!

Thanks,
Rudy.