Topic: How to aggressively prioritize a port with QOS

The content of this topic has been archived on 4 May 2018. There are no obvious gaps in this topic, but there may still be some posts missing at the end.

Page 1 of 1

Post #1

spacediver

26 Oct 2016, 16:35

Note: I couldn't figure out how to include images. When I tried using the [img]url[/img] tag, I got an error saying I'm not allowed to post links. So I've replaced h-t-t-p with xttp:, and if you want to see the images, you'll have to copy and paste the links and change the x to an h. Sorry for the inconvenience.

A couple years ago, I switched from a linksys wrt54gl with Tomato to an TP-LINK Archer C7 with OpenWRT.

With Tomato, the QOS implementation allowed me to prioritize a set of ports very effectively, so that I could play Quakelive without lag. I was able to test this by looking at the lagometer/netgraph display in quakelive, which shows a realtime visualization of latency. A flat line, on this display, is ideal. Any sudden changes in latency is immediately obvious because spikes/bumps appear on the graph. With Tomato, I could send a large file over email, or run an upload test on a website, and it would not have any impact whatsoever on the lagometer, and gameplay was incredibly smooth.

Using Luci-app-qos on the newer router, I'm able to achieve only moderately effective reduction of jitter caused by high upstream usage.

I've experimented with a few different configurations, including editing the qos configuration file directly, but I'm not having much success.

Here is what the Luci settings are:

xttps://i.imgsafe.org/0d034950e1.png (replace x with h)

And here is what the QOS file looks like, with my custom edits highlighted in green.

xttps://i.imgsafe.org/0d07e31977.png (replace x with h)

Note that in the Luci app, I have only defined the ports for which I want high priority. I haven't defined ports that I want normal, or lower priority. However, in the QOS file, I've adjusted the settings for the Normal class, which I am assuming will be applied to all traffic that doesn't fall within the class that I have defined.

Am I missing something essential here? Or is what I'm trying to do simply not possible? I'd appreciate any help!

(Last edited by spacediver on 26 Oct 2016, 16:49)

Post #2

dl12345

26 Oct 2016, 22:11

You could try using SQM scripts (simple.qos). Then in the mangle tables, QOS_MARK_<interface name> table add a rule something like

iptables -t mangle -A QOS_MARK_eth0 -p tcp -m multiport --dports <list of ports> -j MARK --set-mark 0x1/0xff

Adjust for --sports or -p udp or whatever the case may be, This will mark the packets and the tc filters will place your latency sensitive traffic in the first bucket, thereby prioritizing it.

Unfortunately this kind of customization needs to be done outside of luci, so there's no web UI to help you through it.

Post #3

spacediver

27 Oct 2016, 00:00

Thanks for the response. I appreciate it.

I went ahead and uninstalled luci-app-qos and qos-scripts, and installed luci-app-sqm.

Using the Luci interface, I set the ingress and egress to the appropriate values (about 80 percent of actual speeds). I left everything else at defaults (including queue discipline, where the queue setup script was set to simple.qos).

I then connected to the router using putty, and typed vi /usr/lib/sqm/simple.qos, and added the following line:

ipt -t mangle -A QOS_MARK_eth0 -p all -m multiport --dports 27900-27999 -j MARK --set-mark 0x1/0xff

I made sure to save the file, and then enabled SQM.

It doesn't seem to be doing anything. Now when I run a speedtest while playing quakelive, the connection lags out completely.

I wonder if I've set the interface name incorrectly (eth0). I'm connected to the router via ethernet cable, but everyone else is connected via wifi.

(Last edited by spacediver on 27 Oct 2016, 00:58)

Post #4

dl12345

27 Oct 2016, 09:38

Are you sure the iptables rule is getting added correctly?
Does the QOS_MARK_<interface_name> table exist
Is your outgoing interface actually named eth0?

Perhaps you could post the output of an ifconfig and also an iptables -t mangle -vnL.

I also wouldn't edit the simple.qos file to add the iptables rule. It could be more appropriately added via /etc/firewall.user or in a hotplug script in /etc/hotplug.d/iface

Assuming you have some specific shell script to add and remove the iptables rules in /etc/firewall.user.qos, then you'd call this script from /etc/firewall.user and you can also reference this script from a hotplug file, say /etc/hotplug.d/iface/99-qos (or something similar, as long as it gets executed after the corresponding sqm script which is called 11-sqm)

#!/bin/sh

if [ "$ACTION" = ifup  && $DEVICE == <wan_device_name> ]; then
    /etc/init.d/sqm enabled && {
        /etc/firewall.user.qos
        logger "Custom QOS restarted due to ifup of interface $DEVICE"
    }
fi

Post #5

spacediver

27 Oct 2016, 16:23

dl12345 wrote:

Are you sure the iptables rule is getting added correctly?

I'm not sure. I do know that when I closed the Putty session, then restarted it and reopened the simple.qos file, the line I had added was still there.

dl12345 wrote:

Does the QOS_MARK_<interface_name> table exist

I'm not sure, in the simple.qos file, here are the first few uncommented lines.

. /usr/lib/sqm/functions.sh

ipt_setup() {

ipt -t mangle -N QOS_MARK_${IFACE}

ipt -t mangle -A QOS_MARK_${IFACE} -j MARK --set-mark 0x2${IPT_MASK_STRING}
# You can go further with classification but...
ipt -t mangle -A QOS_MARK_${IFACE} -m dscp --dscp-class CS1 -j MARK --set-mark 0x3${IPT_MASK_STRING}
ipt -t mangle -A QOS_MARK_${IFACE} -m dscp --dscp-class CS6 -j MARK --set-mark 0x1${IPT_MASK_STRING}
ipt -t mangle -A QOS_MARK_${IFACE} -m dscp --dscp-class EF -j MARK --set-mark 0x1${IPT_MASK_STRING}
ipt -t mangle -A QOS_MARK_${IFACE} -m dscp --dscp-class AF42 -j MARK --set-mark 0x1${IPT_MASK_STRING}
ipt -t mangle -A QOS_MARK_${IFACE} -m tos  --tos Minimize-Delay -j MARK --set-mark 0x1${IPT_MASK_STRING}

dl12345 wrote:

Is your outgoing interface actually named eth0?
Perhaps you could post the output of an ifconfig and also an iptables -t mangle -vnL.

Here's the output of ifconfig (I ran this command once I had logged into router)

root@OpenWrt:~# ifconfig
br-lan    Link encap:Ethernet  HWaddr 14:CC:20:9F:C4:F2  
          inet addr:192.168.1.1  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::16cc:20ff:fe9f:c4f2/64 Scope:Link
          inet6 addr: fdcc:fa71:9bfd::1/60 Scope:Global
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:16064271 errors:0 dropped:0 overruns:0 frame:0
          TX packets:19079902 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:3703850086 (3.4 GiB)  TX bytes:23910961722 (22.2 GiB)

eth0      Link encap:Ethernet  HWaddr 14:CC:20:9F:C4:F3  
          inet addr:192.0.160.148  Bcast:192.0.160.159  Mask:255.255.255.224
          inet6 addr: fe80::16cc:20ff:fe9f:c4f3/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:50987417 errors:0 dropped:0 overruns:0 frame:0
          TX packets:14579353 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:5 
          RX bytes:86011162 (82.0 MiB)  TX bytes:2922165086 (2.7 GiB)
          Interrupt:4 

eth1      Link encap:Ethernet  HWaddr 14:CC:20:9F:C4:F2  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:6952211 errors:0 dropped:17 overruns:2 frame:0
          TX packets:7435736 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:710031901 (677.1 MiB)  TX bytes:185658381 (177.0 MiB)
          Interrupt:5 

ifb0      Link encap:Ethernet  HWaddr CA:6C:00:B6:D2:A1  
          inet6 addr: fe80::c86c:ff:feb6:d2a1/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:3448887 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3448887 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:5 
          RX bytes:3673164397 (3.4 GiB)  TX bytes:3673164397 (3.4 GiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:91017 errors:0 dropped:0 overruns:0 frame:0
          TX packets:91017 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:7529900 (7.1 MiB)  TX bytes:7529900 (7.1 MiB)

wlan0     Link encap:Ethernet  HWaddr 14:CC:20:9F:C4:F0  
          inet6 addr: fe80::16cc:20ff:fe9f:c4f0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2870729 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4527976 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:912280197 (870.0 MiB)  TX bytes:1340917888 (1.2 GiB)

wlan1     Link encap:Ethernet  HWaddr 14:CC:20:9F:C4:F1  
          inet6 addr: fe80::16cc:20ff:fe9f:c4f1/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:6267690 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8706032 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:2316849338 (2.1 GiB)  TX bytes:2197804602 (2.0 GiB)

and the output after running iptables -t mangle -vnL (this was done after reinstalling SQM, so that line I had added earlier was no longer there)

Chain PREROUTING (policy ACCEPT 95004 packets, 79M bytes)
 pkts bytes target     prot opt in     out     source               destination         
  34M   26G fwmark     all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain INPUT (policy ACCEPT 3890 packets, 341K bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 90902 packets, 78M bytes)
 pkts bytes target     prot opt in     out     source               destination         
  33M   26G mssfix     all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain OUTPUT (policy ACCEPT 4223 packets, 2137K bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain POSTROUTING (policy ACCEPT 95125 packets, 81M bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain fwmark (1 references)
 pkts bytes target     prot opt in     out     source               destination         

Chain mssfix (1 references)
 pkts bytes target     prot opt in     out     source               destination         
81769 5027K TCPMSS     tcp  --  *      eth0    0.0.0.0/0            0.0.0.0/0            tcp flags:0x06/0x02 /* wan (mtu_fix) */ TCPMSS clamp to PMTU

Chain qos_Default (0 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 CONNMARK   all  --  *      *       0.0.0.0/0            0.0.0.0/0            CONNMARK restore mask 0xf
    0     0 qos_Default_ct  all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0xf
    0     0 MARK       udp  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0xf0 length 0:500 MARK xset 0x22/0xff
    0     0 MARK       icmp --  *      *       0.0.0.0/0            0.0.0.0/0            MARK xset 0x11/0xff
    0     0 MARK       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0xf0 tcp spts:1024:65535 dpts:1024:65535 MARK xset 0x44/0xff
    0     0 MARK       udp  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0xf0 udp spts:1024:65535 dpts:1024:65535 MARK xset 0x44/0xff
    0     0 CONNMARK   all  --  *      *       0.0.0.0/0            0.0.0.0/0            CONNMARK save mask 0xf0

Chain qos_Default_ct (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 MARK       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0xf tcp multiport ports 27900:27999 MARK xset 0x11/0xff
    0     0 MARK       udp  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0xf udp multiport ports 27900:27999 MARK xset 0x11/0xff
    0     0 CONNMARK   all  --  *      *       0.0.0.0/0            0.0.0.0/0            CONNMARK save mask 0xff
    0     0 MARK       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0xf tcp multiport ports 27900:27999 MARK xset 0x11/0xff
    0     0 MARK       udp  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0xf udp multiport ports 27900:27999 MARK xset 0x11/0xff

I also wouldn't edit the simple.qos file to add the iptables rule. It could be more appropriately added via /etc/firewall.user or in a hotplug script in /etc/hotplug.d/iface

Noted. Sounds easier to directly open the firewall.user file and add the line in there.

(Last edited by spacediver on 27 Oct 2016, 16:25)

Post #6

dl12345

27 Oct 2016, 20:38

The question "Is the iptables rule getting added correctly" and "Does the QOS_MARK_eth0" table exist can only be answered by looking at the output of "iptables -t mangle -vnL", not by looking at the simple.qos script.

Looking at the output of your "iptables -t mangle -vnL", I can see that you appear to be running the old openwrt qos scripts and NOT SQM scripts, since there is clearly no QOS_MARK_eth0 table in the listing.

You need to

# /etc/init.d/qos disable
# /etc/init.d/qos stop
# /etc/init.d/sqm enable
# /etc/init.d/sqm start

(I think the name of the old openwrt qos scripts is /etc/init.d/qos, but make sure this file exists otherwise it's using a variant of this name).

Once you've done this, SQM scripts should be running and enabled so that it autostarts at boot, which is apparently not the case now, which is why you're not seeing any change in latency. Your iptables rule isn't getting added because there's no table to add it to and because the sqm shell script is not getting invoked.

Ensure that SQM has started by doing a "iptables -t mangle -vnL". If you cannot see a table with the name QOS_MARK_eth0, then you haven't started the init.d script.

Create the file /etc/firewall.user.qos. Make it executable. Add the following

#!/bin/sh

PORTS="2345,6789,5099"   # change this to your port list

SQMIF=$(uci get sqm.wan.interface)
SQMTABLE="QOS_MARK_${SQMIF}"
FWMARK="0x1/0xff"

/etc/init.d/sqm enabled && {

    iptables -t mangle -D ${SQMTABLE} -p all -m multiport --dports ${PORTS} -j MARK --set-mark ${FWMARK} 2> /dev/null
    iptables -t mangle -A ${SQMTABLE} -p all -m multiport --dports ${PORTS} -j MARK --set-mark ${FWMARK}

}

Add the following line to the bottom of the file /etc/firewall.user

echo "/bin/sh -c /etc/firewall.user.qos" >> /etc/firewall.user

Create the file /etc/hotplug.d/iface/99-myqos. Make it executable. Add the following

#!/bin/sh

[[ "$ACTION" == "ifup"  ]] && [[ "$DEVICE" == $(uci get sqm.wan.interface) ]] && {

        /bin/sh -c /etc/firewall.user.qos

}

Create the file /etc/hotplug.d/firewall/99-myqos. Make it executable. Add the following

#!/bin/sh


[[ "$ACTION" == "add" ]]  && [[ "$ZONE" == "wan" ]] && [[ "$DEVICE" == $(uci get sqm.wan.interface) ]] && {

    /bin/sh -c /etc/firewall.user.qos

}

You require the last two scripts because SQM will execute hotplug actions that will remove your rule whenever the interface is bounced or the firewall zone is bounced

Post #7

spacediver

27 Oct 2016, 21:25

Thank you so much, you've been extremely generous with your time and energy.

I'll give this a go tomorrow when I get back home, and will report back.

Post #8

spacediver

28 Oct 2016, 23:15

dl12345 wrote:

The question "Is the iptables rule getting added correctly" and "Does the QOS_MARK_eth0" table exist can only be answered by looking at the output of "iptables -t mangle -vnL", not by looking at the simple.qos script.

Good to know.

I followed all of the instructions except for this last one:

dl12345 wrote:

Create the file /etc/hotplug.d/firewall/99-myqos. Make it executable. Add the following

There is no firewall directory in my hotplug.d directory. Should I manually create the directory?

Your approach seems to work extremely well. Now when I run a speedtest, there is absolutely no discernible change in latency in quakelive during the downstream test. With the upstream test, there is still some inteference, but I can almost eliminate this by reducing the egress rate in sqm to a lower value (about a third of the max egress rate). Clearly, if I go this route, I'll only enable SQM during the time that I actually do play quakelive.

If you don't mind, I have three more questions:

1: How could you tell, based on my ifconfig output, that eth0 was the correct interface to use? If there are others who are on my network but are connected via wifi, would I need to repeat these steps somehow for WLAN0 and/or WLAN1?

2: What does 0x1/0xff signify. I gather it somehow marks the packets with high priority, but what exactly is going on?

3: If one wanted to learn this "stuff", what fields of study would be useful? I know how to program in R and Matlab. I suppose being comfortable in linux is an asset (following your instructions in this thread has already forced me to learn some basic stuff in linux). But all this stuff about iptables, etc is very new to me. Is this all the language of networking? Or is this stuff very specific to openWRT?

(Last edited by spacediver on 28 Oct 2016, 23:21)

The discussion might have continued from here.

Page 1 of 1