Ultimate SQM settings: Layer_cake + DSCP marks

So based on @moeller0's post and the fact that it maps well to both Wifi WMM queues, and the queues on cheap TP-Link managed switches, you should probably switch to CS6 for highest priority, CS3 for mid priority, CS0 for normal priority, and CS1 for bulk. By default linux wifi drivers put EF into the mid priority VIDEO queue rather than the high priority VOICE queue, so I don't use it even though it's the "standard" for voice. (This is in my opinion a bug in the default behavior of the linux driver, apparently it can be adjusted in hostapd, but the config is not available to OpenWrt at the moment).

All those AFxx codepoints are fine-details that no one actually is currently using.

@moeller0 I am currently using CS2 as my "best effort" priority, and retag everything on ingress from my router (and output from my desktop machine) because of those Tp-Link switches. Since I'm not using cake that's ok for me, but I wonder if cake shouldn't switch to having CS0 and CS2 in the same best effort tin.

I guess the issue is that there is an old 3bit priority scheme used in some VLAN handling equipment that does weird mapping (see page 40 of http://profesores.elo.utfsm.cl/~agv/elo309/doc/802.1D-1998.pdf and compare with the more recent recommendations in https://en.wikipedia.org/wiki/IEEE_P802.1p)

The recommendations how to interpret the VLAN priority codes changed over time. IMHO we should stick to the most recent version (if at all ;)*). Also I believe that Jonathan's justifications for the placements come from tags seen on real world traffic or seen as recommendation in RFCs.
Now I note that there is a proposal for a new background code point (out of the the CS0 space, so no priority inversion when traffic hits non-compliant hops). Also I note that in standard WMM mapping CS2 becomes background

*) I think I get your point though, even though WMM and VLAN priority schemes might be considered unfortunate, they still exist in real live and it might make sense to take them into account (like the WMMM/EF anomality you mention, also seen in https://www.bintec-elmeg.com/portal/downloadcenter/dateien/workshops/current_en/ws_wlan_html_en_HTML/vowlan_infra_qos_wmm.html) This is a mess :wink:

Oh thanks, very helpful guys! d-^;^-b

I have WMM off on my wifi interface and all but my static IP gets wiped off DSCP marks tho, but i will adhere to these rules anyway and not use EF :stuck_out_tongue:

Probably more laziness than anything. I might change the setup and not do this, im just afraid that something in my iPhone or macbook could potentially supersede my high priority gaming or even slightly affect its UDP datastream I will go nuts! :stuck_out_tongue:

My iphone jumps from different isp/wifi anyway whenever depending on range, and my macbook rarely does anything like ever, other than Local remoting.

One thing that might be consideration is my ISP and how it would handle CS6 vs EF? Its T-mobile 4G in the netherlands, no clue really if they ignore it or not. Especially since 4G is used for voice as well?

No doubt.

Looking at these two sites:

and
https://wireless.wiki.kernel.org/en/developers/documentation/mac80211/queues

I see CS2 = 16, which after multiplying by 4 for the ECN bits, maps to TOS 64, which maps to UP 2 which maps to Background queue!!! jeezus that's right and it's clearly stupid.

But note what happens if you use something like CS3, Cake treats it as "Streaming Media" and WMM treats it as Best Effort!!!

For actual streaming media (YouTube etc) I'm using AF41, which at least gets consistency in cake and WMM

As you say it's a mess. When do we get access to that hostapd priority map config in OpenWrt?

My favorite scheme would be:

CS1 = bulk
CS0, CS2 = Best Effort
CS3,CS4,CS5 = Streaming Media
EF,CS6,CS7 = Voice

and treat other values by just mask off the lower 3 bits. It's very comprehensible.

I would definitely wipe DSCP on ingress from the internet (this is actually kinda how DSCP is supposed to work, it's not an end-to-end specification, there are "diffserve domains" and the traffic is "supposed" to be reclassified at the boundaries, at least that's basically what the RFC was suggesting).

Also on ingress from your wifi, I'd wipe any DSCPs and then maybe just maybe add back ones you can identify as important (like maybe wifi calling or VOIP service packets get CS6). There's no way to avoid the Client -> AP being affected by the DSCP but you can at least affect AP -> client and AP -> internet

And then of course, set your high priority for UDP game packets = CS6... and off you go, this should protect your game stuff properly.

For me it's crappy VOIP calls that I'm really unhappy about, and it's amazing how much better it is with proper QoS, my voice calls are crystal clear these days. Note also that I got bad problems with VOIP even with a supposed gigabit fiber connection. Part of it is that my network can get busy, I run NFS home directories, and if a big computation is running on my desktop it can be slamming data into files at full gigabit speeds for seconds at a time, so I need QoS not just on my WAN but also on my LAN!

I decided to dig up the old thread on qos_map to see if anything has changed in the interim with the version 18 series: Using DSCP for QoS

EDIT:

Hey all, I discovered something interesting....

Setting up tagging and by default tagging everything cs2 seems to have been the cause of my erratic behavior. If I tag cs2 my ATT connection seems to down-prioritize or throttle my traffic (possibly my ACKs?), not sure what, if I tag cs3 as my "normal" it seems to be ok. Basically changing that one line made things go from highly erratic to clean as a whistle. So I've switched to CS3 as my "default" tag.

Also, they symptom was a much slower start on download but reaching a moderately reasonable level, but on upload completely lack of bandwidth eventually resulting in bandwidth dying out almost altogether, which is why I think it has to do with ATT equipment interpreting outgoing DSCPs I send.

More information: tc is dead, long live nftables... Here are rules I use to classify into my multi-tier HFSC shaper:

	  meta priority set 1:40 ## default

	  ip dscp {ef,cs6} meta priority set 1:10
	  ip dscp {cs5} meta priority set 1:20
	  ip dscp {af41, af42, af43} meta priority set 1:30
	  ip dscp {cs1} meta priority set 1:50

	  ip6 dscp {ef,cs6} meta priority set 1:10
	  ip6 dscp {cs5} meta priority set 1:20
	  ip6 dscp {af41, af42, af43} meta priority set 1:30
	  ip6 dscp {cs1} meta priority set 1:50

This replaces about 40 lines of more complicated tc filter commands that look like:

 /sbin/tc filter add dev ${DEV} parent 1:0 protocol ip prio 60 u32 match ip tos 0x88 0xfc flowid 1:30
 /sbin/tc filter add dev ${DEV} parent 1:0 protocol ipv6 prio 61 u32 match ip6 priority 0x88 0xfc flowid 1:30 

some of which included using tc to match tcp / udp ports instead of DSCP values.

All I need now is tc to set up the hfsc shaper itself, which is much more understandable than the u32 filter syntax.

I think nftables provides the ideal comprehensive method for QoS afficionados (gamers, Voipers, guest network providers etc) who want more control than the very good defaults that piece-of-cake offers.

1 Like

@dlakelan nice work man!, i was busy trying to figure out why i can't unlock my phone bootloader!
it's really much easier to setup and understand nft qos.
so basically my idea about bridge is working well for you?!, i think it's possible to use for those who have
a different rate for ipv6, as some people get ipv6 connection with dhcpv6 or 6in4, ...etc.
by setup another veth pair and bridge one of them with wan and use second one to get a connection!
the question is:
1.when nft will be natively used on openwrt instead of iptables.

i think we still need tc to setup shaper, unless nft came with alternative solution.
it would be nice to see a packet inspection or L7 in nft!.
*i figured that my isp is zeroing dscp on inbound, on old isp when i was watching youtube i see AF41 tag, and
AF31 on a filehost site!
@Emtee
nice to see that you happy with your qos, for testing purpose change your download speed in sqm to 40000.
then run a download and play a game, then see how's the lag!

Yes when I put it in it's own namespace, I think that's key. Then veth1 looks like a wan connection to an ISP that has a clue :grinning:

1 Like

did you use an external power supply to power this mother board,and how much dose power supply cost?!
also do you buy a case for the MB, or you just place it inside a cartoon box?
also i can use one Ethernet as wan and second one as lan to connect it to wifi AP

This is a good question. Since nftables has its own special syntax for rules and that syntax is excellent, it seems like trying to shoehorn that into a UCI based thing would be problematic. I mean, it could be done, but would not give the advantages of the nice syntax.

In any case, I'm going to look into building an image for my test device (an old tp-link device) that has no iptables no OpenWrt firewall no luci-app-firewall and does include tc and nftables with a default config included for both. If I can make that work it will be my new AP image. Still not quite sure how to make it start nftables at the appropriate time during boot.

Yes it's in a small case. This one with power supply I've had it for years
https://www.amazon.com/dp/B00L5OT1QY/ref=cm_sw_r_cp_apa_i_rK9dCbAHYFMZH

I use an asrock rack j1900d2y but I'd recommend something newer if you plan to buy a router, something with a quad core 3000 series Celeron with aes-ni. Lots of stuff on AliExpress...

i plan to use it as router!
but is it necessary to use display monitor to install and configure the OS, cause i don't have display monitor.
i saw that tp-link archer c7 AC1750 is cost about 95$ i my country, so it's better to buy mother board or mini pc
with modern Intel CPU!
i think this one is good, but it's with only one lan port!: https://www.scan.co.uk/products/asrock-j4005m-integrated-intel-celeron-j4005-ddr4-sata-iii-intel-uhd-graphics-600-gbe-microatx
but if i want to search on AliExpress what should i type!?

"mini pfsense pc" would get you started they tend to advertise that way

Since you don't have a monitor, you might want to look at something that supports IPMI or has a serial console. a bunch of those aliexpress devices do have serial consoles.

1 Like

@dlakelan
Hello
I was trying to prioritize TCP-flags, so i have used this:
$IPT -t mangle -A PREROUTING -p tcp -m tcp ! --tcp-flags ALL ACK -j DSCP --set-dscp-class CS3
i think this will prioritize all tcp-flags except ACK, is this right?

and if i want prioritize ACK based on length, i should use something like this?
$IPT -t mangle -A PREROUTING -p tcp -m tcp --tcp-flags ALL ACK -m length --length :64

  • i forget to mention that iptables doesn't support ECE and CWR flags?!
    cause for normal internet browsing, etc ACK size will be small, and for large transfers like torrent it will be
    bigger,so i have to set it as bulk?

*strange thing when i set TOS bits for ICMP packets and other DSCP marks, i see my ping is lower; from
70ms to 50ms, i think my isp is honoring those tags, also i saw that NTP is tagged with AF21 by isp !
I used this table https://wiki.innovaphone.com/index.php?title=Howto:Calculate_Values_for_Type_of_Service_(ToS)_from_DiffServ_or_DSCP_Values to set TOS!

I'm not sure it might prioritize all tcp packets except those with ACK flag, including those with no flags. The sense may be different if the ! Comes after the --tcp-flags vs before.

it's hard to understand how it's work, i read the man page and on other pages, but still can't understand it
correctly, but iptables will not accept ! after --tcp-flags, i don't know why.

$IPT -t mangle -A PREROUTING -p tcp -m tcp --tcp-flags ! ALL ACK -j DSCP --set-dscp-class CS3
iptables v1.6.2: Unknown TCP flag `!'

Yeah, I think the man page I was reading may have been out of date.

I think your original thing will prioritize any packet except packets that have just ACK flag, which includes non-flag packets, and packets that are ACK together with other flags, like ACK,PSH or SYN,ACK or SYN,ACK,PSH etc

Not clear what you really want to do. what is the goal?

i want to prioritize all tcp-flags like fin,psh,rst,syn,urg except ACK to optimize network response, when there's
a network load, then i will prioritize small ACK's based on length except other flags!.
cause ACK sometimes is big, especially when you have torrent's is running.

I would be very careful with selectively boosting any sub-group packets of a given to flow as that might introduce packet reordering, which is decidedly not nice to have. Treating sparse flies differently (as both cake and fq_codel do) does not really suffer from this issue so much as for a typical down-to-earth or upload one direction (the one with the ACKs) will be sparse, while the other is not.
So I am not saying your idea does not have merit, but rather that you should test it extensively to see that it is free of undesired side-effects.
Best Regards

Thanks @moeller0
i assume that you recommend that i don't prioritize those tcp-flags, so it's better to depend on
cake treating.

I understand that, but can you suggest an idea to make network response better?!
i was reading here: https://community.ubnt.com/t5/EdgeRouter/iptables-PRIO-queues-different-QOS-approach/m-p/805034#U805034
and i tried to get some useful rules!, also why no one is using CS7. is CS7 have higher priority than CS6?
also when i set a TOS along with DSCP to packets, i saw that network response is better, maybe my isp
is honoring TOS?
cause i see that NTP packet coming from ISP with AF21 tag.

Thanks and Best Regards.