Ultimate SQM settings: Layer_cake + DSCP marks

dlakelan · 2019-01-04T15:37:26.064Z

Stats on wmm usage are interesting how did you get that?

hisham2630 · 2019-01-04T16:02:29.783Z

hehehe, too simple

 cat /sys/kernel/debug/ieee80211/phy1/ath9k/xmit
 cat /sys/kernel/debug/ieee80211/phy1/ath9k/queues

and this one for errors and frags:

cat /sys/kernel/debug/ieee80211/phy1/ath9k/recv

can you have a look at post

dlakelan · 2019-01-04T16:36:31.799Z

hisham2630:

i'm planing to change my tags like that, correct me if it's wrong:

CS6 for voip/gaming

CS5 for dns and icmp

CS2 some tcp flags and

AF21 another set of tcp flags

AF31 for youtube and other streaming

CS0 for none prioritized traffic, also mail ports?!

AF11 internet browsing and games login?

CS1 for downloads and torrents

Diffserv8 may have 8 queues, but that doesn't mean you need to use all of them. I'd limit it to

Realtime stuff, tagged CS6 (VOIP, games, NTP)
Low Latency small stuff tagged CS5 (flags, DNS)
Video Streaming tagged AF41 (this matches with WMM Video queue)
Best Effort Stuff tagged CS0 (goes into BE WMM queue)
Background CS1 (Goes into BK WMM queue)

Beyond that trying to use other tags will wind up getting you conflicts between cake and WMM and some cheap managed switches and various other things that might use DSCP, one thing will treat say CS2 as prioritized, another one will treat it as Background, etc.

hisham2630 · 2019-01-04T17:45:57.883Z

Thanks.
I see that when 2 clients connected to the same lan, assume client A call client B using viber voice call app
viber will tag packets with AF41 for both outbound and inbound, also the call is p2p using the private ip of
those 2 clients.
My ISP is tagging NTP packets with AF41 ?!

dlakelan · 2019-01-04T19:37:08.693Z

Since there are no hard standards each app or provider uses their own system, it's a bit of a mess. But af41 is reasonable as a tag for low latency data. It won't go into voice WMM queue though

hisham2630 · 2019-01-05T15:28:21.186Z

I saw there's something wrong, can you catch it?! @dlakelan Screenshot%20(2)
Screenshot%20(5)
Screenshot%20(6)
There's an interested things here

Emtee · 2019-01-05T19:13:13.295Z

Hi again guys! Perhaps you can help me extend this setup. (if any1 has any other tips to optimize tcp over openvpn be my guest)

I'm trying to add OpenVPN to this for my game and im planning to route the traffic from my windows through the OpenVPN client on OpenWRT by destination IP, or policy based routing)

I tried Windows VPN clients with split-tunneling to try and only route specific apps through VPN but it seem mostly dirty fixes with firewall rules i believe, and just feels dirty.

Currently the OpenVPN works (tun0) and connection established, turned off nagling with tcp-nodelay.

The default routes work to the clients depending whether i use 'route-nopull' or not, obviously use it as i want to do some manual route.

But first I gotta get the Tun0 to work with the veth setup so also ingress from tun0 goes through veth!

Thank you!

dlakelan · 2019-01-05T20:20:00.651Z

with a vpn you'll have two kinds of ingress and two kinds of egress, one "into / out of the tunnel" and one the actual encrypted packet sent out or in on ethernet. If you prioritize the unencrypted (pre/post tunnel) packet, but the encrypted packet waits... you won't get the prioritization you want. It's a bit tricky because you can't look at the encrypted packet at all so likely you want to restrict the tunnel to just gaming, and prioritize all the encrypted packets.

also, if you can use a VPN provider who uses wireguard, it is a better choice due to lower CPU requirements for the router.

Emtee · 2019-01-05T20:37:40.158Z

Yep, that will be hard to prioritize inside tunnel too, but... the VPN is intended for solely one game! I know the IP destination range, but im having trouble setting it up in the same veth setup as @hisham2630

So at least the tunnel should not be hard to prioritize now and I can just prioritize the whole port of my choosing, i think? (Currently just port 443, but I can change to a different one later)

If I can just have both Eth1.2 (WAN) and Tun0 both let egress go over Veth0/1, and the tunnel sends egress back the regular route to Eth1.2 (WAN)

Or is this too simple thinking?

Edit: I think I already managed it working, but not sure if OK. When I do speedtest (Download) i can see equal Tun0 and Veth data flow!

What i did:

ip route add default dev veth0 table 100
ip rule add iif eth1.2 table 100 priority 100
ip rule add iif tun0 table 100 priority 100

Just added the last line. I use this package to specify route atm the easy way:

VPN Policy-Based Routing + Web UI -- Discussion Community Builds, Projects & Packages

VPN Policy-Based Routing is a service supporting multiple types of VPN Connections (Openconnect, OpenVPN, PPTP and Wireguard) allowing you to create policies to use either VPN tunnel or WAN as a gateway. More information (requirements, full features list, etc.) on the service is available in the README. Gateways/Tunnels Any policy can target either WAN or a VPN tunnel interface. L2TP tunnels supported (with protocol names l2tp*). Openconnect tunnels supported (with protocol names openconne…

dlakelan · 2019-01-05T20:45:05.621Z

I think you want your new rule to have different priority, otherwise you might be overwriting your other rule, not sure. Try:

ip route add default dev veth0 table 100
ip rule add iif eth1.2 table 100 priority 100
ip rule add iif tun0 table 100 priority 101

easyteacher · 2019-01-06T01:31:34.625Z

ipt="iptables -t mangle"
$ipt -A mark_DSCP_ul \
		-p udp \
		-m multiport --dports 1024:65535 \
		-m connmark ! --mark 0x30/0xf0 \
		-m connbytes \
			--connbytes :128 \
			--connbytes-dir original \
			--connbytes-mode avgpkt \
		-m connbytes \
			--connbytes :512 \
			--connbytes-dir reply \
			--connbytes-mode avgpkt \
		-m hashlimit \
			--hashlimit-above 15 \
			--hashlimit-mode srcip,srcport \
			--hashlimit-srcmask 32 \
			--hashlimit-name "mark_game_above" \
		-m hashlimit \
			--hashlimit-upto 128 \
			--hashlimit-mode srcip,srcport \
			--hashlimit-srcmask 32 \
			--hashlimit-name "mark_game_upto" \
		-j CONNMARK --set-xmark 0x50/0xf0 \
		-m comment --comment "Realtime Interactive: Games, match upload traffic, tickrate 15(PUBG)-128(CSGO)"

$ipt -A PREROUTING -m conntrack --ctstate RELATED,ESTABLISHED -m connmark ! --mark 0xf0/0xf0 -m connmark ! --mark 0x60/0xf0 -m connmark ! --mark 0x70/0xf0 -m statistic --mode random --probability 0.005 -j CONNMARK --and-mark 0xff0f
$ipt -A PREROUTING -i br-lan -m connmark --mark 0x0/0xf0 -j mark_DSCP_ul

$ipt -A PREROUTING -m connmark --mark 0x50/0xf0 -j DSCP --set-dscp-class CS4

My iptables rule to match the game traffic. Connmark, connbytes and hashlimit are used. LOL tested.

hisham2630 · 2019-01-06T09:32:55.737Z

It would be nice if you can get it to work.you can use conntrack to auto tag packets on both ways /in/out.

hisham2630 · 2019-01-06T10:39:30.601Z

Nice to see something new, it's look complicated, but interested

i just use those two rules to catch voip/games, it's working good, many thanks -j @dlakelan

$IPT -t mangle -A PREROUTING -p udp -m hashlimit --hashlimit-name udp_high_prio --hashlimit-above 120/sec --hashlimit-burst 50 --hashlimit-mode srcip,srcport,dstip,dstport -j CONNMARK --set-mark 0x55 -m comment --comment "connmark for udp"
$IPT -t mangle -A PREROUTING -p udp -m connmark ! --mark 0x55 -m conntrack --ctorigsrc 192.168.1.0/24 -m multiport ! --ports 53,5353,80,443,8080,60887 -m connbytes --connbytes 0:700 --connbytes-dir both --connbytes-mode avgpkt -j DSCP --set-dscp-class CS6 -m comment --comment "small udp connection"

It's tag packets on both in/out bound.
Also which rules do you use to catch bulk traffic, and which rules to speedup web browsing!

Emtee · 2019-01-06T14:26:46.822Z

This is interesting stuff, I wonder though how accurate is this? I mean is it possible not always game packets are intercepted and tagged?

I theory you could just prioritize traffic on how it behaves / footprint etc, but doesnt piece_of_cake pretty much do this already sufficiently?

hisham2630 · 2019-01-06T14:51:35.361Z

Usually games and VOIP doesn't max out packets size, it's use a small packet size to handle important sync
data or voice data, we do exclude port 80,443 http/https, conntrack will catch dest port and ip.

Emtee:

piece_of_cake

piece_of_cake will place the whole traffic inside one tin, also tagging packets will be useless in this case.
layer_cake will honor your dscp tags and will place the in appropriate tin, you can use 3 or 4 or even 8 tins.
they called diffserv3,diffserv4 and diffserv8.

Emtee · 2019-01-06T14:54:42.458Z

Yea true, gaming packets and voip usually dont max out the payload. Perhaps just tag packets based on its payload size would actually work well with QoS cake? Dunno, just thinking out loud here

I'm just wondering how you could make general rules and optimize it all for general use without having to resort to specifically aiming at applications/games separately.

hisham2630 · 2019-01-06T14:57:40.972Z

This will make rules more effective also you will end have a few rules to maintain and understand.more rules
will use more cpu and maybe will make it useless.
it's just the knowledge and testing (try/error), i do packet capturing to see how it's work!

Ernieelias · 2019-01-06T14:58:18.876Z

Hello everyone! I came across this post not too long ago and read a couple of the posts in the thread and I have became very interested in setting up dscp markings for my network. But I have no idea whatsoever on how to begin, like which packages to download and how to start marking the packets. Please if you guys can help me out on setting up dscp marking I would really appreciate it. And thanks @dlakelan and @hisham2630 for helping everyone out and contributing to help people achieve the best latency in their networks! Keep up the awesome work!

dlakelan · 2019-01-06T16:56:06.047Z

Hi @Ernieelias welcome to the party first thing you should do is install SQM and tune it to your connection speed. Already this will go a long way toward making games and VOIP work better. Everything else here is second order improvement.

Start by running a speed test on dslreports and post the link to the results page here. Then using the Luci app for sqm enable an sqm on your wan interface with about 90% of measured download speed and 95% of measured upload. Redo the test and post that. Once you are at that point we can discuss next steps.

Ernieelias · 2019-01-06T17:20:21.050Z

Thanks! Ah yes sorry, I have forgot to mention that I do have sqm installed on Wrt3200acm and I have a wrt1900acv2 setup as a dumb ap. Did you still want me to post the settings of sqm?