As to my original question, is tagging web browsing traffic not possible? Surely tagging anything using ports 80 and 443 should be quite trivial.
Thanks for this.
How would I know if I can split up my network this way?
of course you can dscp mark 443... ( no guarantee it's web browsing tho' )...
As I understand it, it depends on the architecture of your router hardware. The problematic case is where you have both ethernet and wifi LAN interfaces on the router. The veth workaround essentially allows you to shape both interfaces as one, by placing a virtual interface in front of them and putting the SQM instance on that.
In my case I don't have to do that because my router doesn't handle wifi - I have separate wifi access points which are wired into the LAN elsewhere.
More info from far more knowledgeable people than me can be found in this thread:
I don't use the WiFi on my TP-LINK Archer C7 at all, I have Plumes in bridge mode performing those tasks.
Ok, you should be able to do something like me then. Even simpler if you don't have a guest VLAN.
You need to split your SQM instance in half - egress on the WAN interface to handle upload traffic, and egress on the LAN interface to handle download traffic.
And in between them you have your iptables rules which mark packets with DSCP tags before they hit the SQM instances.
Loads of info and ideas in this thread:
Okay, amazing - and are you able to provide the script you use for your side and to help me set it up? I would be very grateful.
What I am not clear about, is if I put BitTorrent traffic into low effort bin say, does this mean BitTorrent is always limited to low bandwidth or is this only the case if the connection is saturated?
My script is pretty complicated with separate chains to do the tagging and connmarking and loads of aliases which simplify things for me, but would probably confuse anyone else!
It works out as something like this though:
# Tag outgoing Bittorrent packets (and mark their connections) as DSCP CS1 (8) (low priority).
# My Bittorrent traffic uses outbound ports 46792 and 46800-46849.
iptables -t mangle -A PREROUTING -p tcp ! -d 192.168.0.0/16 -m multiport --sports 46792,46800:46849 -j DSCP --set-dscp 8
iptables -t mangle -A PREROUTING -p udp ! -d 192.168.0.0/16 -m multiport --sports 46792,46800:46849 -j DSCP --set-dscp 8
iptables -t mangle -A PREROUTING -p tcp ! -d 192.168.0.0/16 -m multiport --sports 46792,46800:46849 -j CONNMARK --set-mark 8
iptables -t mangle -A PREROUTING -p udp ! -d 192.168.0.0/16 -m multiport --sports 46792,46800:46849 -j CONNMARK --set-mark 8
# Tag incoming Bittorrent packets (and mark their connections) as DSCP CS1 (8) (low priority).
# My Bittorrent traffic uses incoming port 46792 on the router.
iptables -t mangle -A PREROUTING -p tcp -m addrtype --dst-type LOCAL ! -d 192.168.0.0/16 -m multiport --dports 46792 -j DSCP --set-dscp 8
iptables -t mangle -A PREROUTING -p udp -m addrtype --dst-type LOCAL ! -d 192.168.0.0/16 -m multiport --dports 46792 -j DSCP --set-dscp 8
iptables -t mangle -A PREROUTING -p tcp -m addrtype --dst-type LOCAL ! -d 192.168.0.0/16 -m multiport --dports 46792 -j CONNMARK --set-mark 8
iptables -t mangle -A PREROUTING -p udp -m addrtype --dst-type LOCAL ! -d 192.168.0.0/16 -m multiport --dports 46792 -j CONNMARK --set-mark 8
# Identify any packets in any direction on connections marked with DSCP CS1 (8), and apply the DSCP tag.
iptables -t mangle -A PREROUTING -m connmark --mark 8 -j DSCP --set-dscp 8
Low priority traffic will only be guaranteed a very small amount of bandwidth, but it can use all the bandwidth if nothing else is using it.
2 Likes
Due to VDSL2/PTM's 64/65 encoding you might be able to run cake up to
64.199 * 64/65 = 63.211 Mbps
7.999 * 64/65 = 7.876 Mbps
Maybe a bit less if G.INP is in use, so 53.0/7.7 should be o problem for your sync, assuming that BT does not use a traffic shaper at their side. Could you please post a speedtest with SQM disabled (and one just after that with SQM enabled again? To get an idea what kind of bandwidth your link can actualy deliver.
Could you log into your router via ssh and run the following commands and copy and paste the output here, please:
cat /etc/config/sqm
ifstatus wan
tc -s qdisc
(I know you posted tc -s qdisc above, but it would be nice to see this again with the config sqm contents).
That is somewhat easier to look at than screen shots 
What packages does one need to do this? Some connmark things etc i guess?
I use these rules in my Torrent box to mark all traffic of the user 'transmission' as bulk. It works great with SQM in the router.
# firewall-cmd --permanent --direct --add-rule ipv4 mangle OUTPUT 1 \
-m owner --uid-owner transmission \
-j DSCP --set-dscp-class CS1
# firewall-cmd --permanent --direct --add-rule ipv6 mangle OUTPUT 1 \
-m owner --uid-owner transmission \
-j DSCP --set-dscp-class CS1
These are the packages I use - some of them are installed by default I think:
dnsmasq-full
ipset
iptables-mod-conntrack-extra
iptables-mod-extra
iptables-mod-hashlimit
iptables-mod-ipopt
1 Like
G.INP is enabled.
SQM disabled:
SQM enabled:
These were both taken on a wired connection.
root@OpenWrt:~# cat /etc/config/sqm
config queue 'eth1'
option debug_logging '0'
option verbosity '5'
option qdisc 'cake'
option linklayer 'ethernet'
option overhead '34'
option qdisc_advanced '1'
option squash_dscp '1'
option squash_ingress '1'
option ingress_ecn 'ECN'
option egress_ecn 'NOECN'
option qdisc_really_really_advanced '1'
option upload '7700'
option script 'layer_cake.qos'
option iqdisc_opts 'nat diffserv3 dual-dsthost ingress'
option eqdisc_opts 'nat diffserv3 dual-srchost'
option download '50000'
option interface 'pppoe-wan'
option enabled '1'
root@OpenWrt:~# ifstatus wan
{
"up": true,
"pending": false,
"available": true,
"autostart": true,
"dynamic": false,
"uptime": 336753,
"l3_device": "pppoe-wan",
"proto": "pppoe",
"device": "eth0.2",
"updated": [
"addresses",
"routes"
],
"metric": 0,
"dns_metric": 0,
"delegation": true,
"ipv4-address": [
{
"address": "81.156.112.22",
"mask": 32,
"ptpaddress": "172.16.14.147"
}
],
"ipv6-address": [
{
"address": "fe80::1dd8:109:c476:4b2d",
"mask": 128
}
],
"ipv6-prefix": [
],
"ipv6-prefix-assignment": [
],
"route": [
{
"target": "0.0.0.0",
"mask": 0,
"nexthop": "172.16.14.147",
"source": "0.0.0.0\/0"
}
],
"dns-server": [
"81.139.56.100",
"81.139.57.100"
],
"dns-search": [
],
"inactive": {
"ipv4-address": [
],
"ipv6-address": [
],
"route": [
],
"dns-server": [
],
"dns-search": [
]
},
"data": {
}
}
root@OpenWrt:~# tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
Sent 24649931489 bytes 130049623 pkt (dropped 0, overlimits 0 requeues 2)
backlog 0b 0p requeues 2
maxpacket 1514 drop_overlimit 0 new_flow_count 17 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
Sent 230803997023 bytes 201583501 pkt (dropped 0, overlimits 0 requeues 17)
backlog 0b 0p requeues 17
maxpacket 1506 drop_overlimit 0 new_flow_count 33 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br-lan root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev eth1.1 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev eth0.2 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc cake 8067: dev pppoe-wan root refcnt 2 bandwidth 7700Kbit diffserv3 dual-srchost nat nowash no-ack-filter split-gso rtt 100.0ms noatm overhead 34
Sent 28747329 bytes 45358 pkt (dropped 1863, overlimits 41248 requeues 0)
backlog 0b 0p requeues 0
memory used: 140864b of 4Mb
capacity estimate: 7700Kbit
min/max network layer size: 29 / 1492
min/max overhead-adjusted size: 63 / 1526
average network hdr offset: 0
Bulk Best Effort Voice
thresh 481248bit 7700Kbit 1925Kbit
target 37.9ms 5.0ms 9.5ms
interval 132.9ms 100.0ms 104.5ms
pk_delay 0us 730us 802us
av_delay 0us 197us 43us
sp_delay 0us 8us 8us
backlog 0b 0b 0b
pkts 0 47124 102
bytes 0 31518008 11141
way_inds 0 179 0
way_miss 0 1813 17
way_cols 0 0 0
drops 0 1863 0
marks 0 0 0
ack_drop 0 0 0
sp_flows 0 4 1
bk_flows 0 2 0
un_flows 0 0 0
max_len 0 1492 305
quantum 300 300 300
qdisc ingress ffff: dev pppoe-wan parent ffff:fff1 ----------------
Sent 56354682 bytes 57438 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc cake 8068: dev ifb4pppoe-wan root refcnt 2 bandwidth 50Mbit diffserv3 dual-dsthost nat wash ingress no-ack-filter split-gso rtt 100.0ms noatm overhead 34
Sent 55816070 bytes 57077 pkt (dropped 361, overlimits 68511 requeues 0)
backlog 0b 0p requeues 0
memory used: 249984b of 4Mb
capacity estimate: 50Mbit
min/max network layer size: 32 / 1492
min/max overhead-adjusted size: 66 / 1526
average network hdr offset: 0
Bulk Best Effort Voice
thresh 3125Kbit 50Mbit 12500Kbit
target 5.8ms 5.0ms 5.0ms
interval 100.8ms 100.0ms 100.0ms
pk_delay 202us 93us 12us
av_delay 5us 15us 0us
sp_delay 5us 7us 0us
backlog 0b 0b 0b
pkts 51 57381 6
bytes 8148 56345914 620
way_inds 0 153 0
way_miss 39 1864 4
way_cols 0 0 0
drops 0 361 0
marks 0 0 0
ack_drop 0 0 0
sp_flows 1 3 1
bk_flows 0 1 0
un_flows 0 0 0
max_len 597 1492 152
quantum 300 1514 381
Yes. Personally I have all my SQM related rules/commands in a separate script and launch that from the custom rules script, but it amounts to the same thing.
I've got lots of classification rules so I've optimised my setup by ensuring that only the first packet on each connection goes through the full ruleset. Packets on established connections go through a much smaller set of reclassification rules, which downgrade the priority of large downloads on web ports and things like that.
1 Like
Thanks. So the 53 Mbps without SQM results indicates, that at least towards netflix's CDN nodes you do not see goodput commensurate with your Sync. That said your selected 50 Mbps for SQM should still work, and:
50 * ((1500-8-20-20)/(1526)) = 47.575 Mbps
seems close enough to the measured 46 Mbps (which due to rounding can be between [45.5 and 46.5[) to be acceptable (at least to me 
). So things seem to be configured okayish.
Now on to your network, I assume you only operate a single WiFi router and all machines connect to that directly, so no additional router or AP in use?
Specifically, with your torrent versus browsing tests, were both machines connected via cables? The question is basically is the issie caused by sqm on the wan link or is it a secondary wifi issue?
Now, back to your
The IP profile for my line - just tested - is 56Mbps so 53Mbps throughput would seem to tally up with that reasonably well. I am using PPPoE.
On my network, I have a TP-LINK Archer C7 which performs the routing and DHCP and then I have a separate mesh network via Plume which connects to the Archer C7. That provides all of the Wi-Fi in the house, hence the Wi-Fi on the Archer C7 is disabled and not in use.
For the torrenting test, one machine is connected via a cable and the other via Wi-Fi.
But I ran a ping from the router directly and I could see ping spikes upwards of 30ms+ (8ms is my unloaded ping). So possibly not enough on its own to cause an issue but I think this confirms Wi-Fi isn't entirely at fault.
Mmmh, if you switch roles between the browsing and torrenting machine, does the latency effect stay the same?
And for the latency test from the router, could you try to run something like mtr -ezb4 8.8.8.8 while performing your test, with different endhosts to see whether under load it is only the RTT to the emd host that increases or whether you see an RTT "step" along the path?
Yes, I can see the same latency effect, with some spikes upwards, up to 64ms I saw on this test run.
I ran the command as you suggested, this is during the torrent download.
My traceroute [v0.85]
OpenWrt (0.0.0.0) Sun Oct 11 14:38:15 2020
@Not a TXT recordplay mode Restart statistics Order of fields quit
@Not a TXT record Packets Pings
Host @Not a TXT recordLast Avg Best Wrst StDev
17 6.2 9.6 5.8 52.2 11.0
AS2856 31.55.186.177 (31.55.186 93.8% 17 7.5 7.5 7.5 7.5 0.0 6.2 6.2 0.0 3. AS2856 31.55.186.176 (3117 11.1 12.2 1 48.7 9.7 7.9AS2856 host213-121-192-70.ukcor 0.0% 17 9.4 16.5 7.7 106.1 23.9
5. AS2856 peer2-et0-1-3.slough.ukc 0.0% 17 9.4 22.6 8.7 128.6 32.0
6. AS2856 109.159.253.219 (109.159 0.0% 17 10.2 17.6 8.5 127.9 28.8
7. AS15169 216.239.62.75 (216.239.6 0.0% 16 11.8 24.2 10.2 211.8 50.0
8. AS15169 64.233.175.107 (64.233.1 0.0% 16 9.7 18.9 8.2 139.8 32.6
9. AS15169 dns.google (8.8.8.8) 0.0% 16 11.6 17.0 8.5 125.2 28.9
OpenWrt (0.0.0.0) Sun Oct 11 14:38:15 2020
@Not a TXT recordplay mode Restart statistics Order of fields quit
@Not a TXT record Packets Pings
Host @Not a TXT recordLast Avg Best Wrst StDev
17 6.2 9.6 5.8 52.2 11.0
AS2856 31.55.186.177 (31.55.186) 93.8% 17 7.5 7.5 7.5 7.5 0.0 6.2 6.2 0.0
3. AS2856 31.55.186.176 (31 17 11.1 12.2 1 48.7 9.7 7.9
AS2856 host213-121-192-70.ukcor 0.0% 17 9.4 16.5 7.7 106.1 23.9
5. AS2856 peer2-et0-1-3.slough.ukc 0.0% 17 9.4 22.6 8.7 128.6 32.0
6. AS2856 109.159.253.219 (109.159 0.0% 17 10.2 17.6 8.5 127.9 28.8
7. AS15169 216.239.62.75 (216.239.6 0.0% 16 11.8 24.2 10.2 211.8 50.0
8. AS15169 64.233.175.107 (64.233.1 0.0% 16 9.7 18.9 8.2 139.8 32.6
9. AS15169 dns.google (8.8.8.8) 0.0% 16 11.6 17.0 8.5 125.2 28.9
Mmmh, looking at the worst case RTT, the issue seems to happen after/upstream of your router. If true this would indicate that SQM is not going to be able fix your latency issues... since it is not in control of the true bottleneck.
Maybe you can repeat this test with different end points for mtr, like e.g. 1.1.1.1 and 9.9.9.9 (other anycasted DNS servers that hopefully resolve to instances in CDNs nearby).
Also, it might be interesting to repeat this test with the sqm set to 25 Mbps.