SQM 3 layer cake with br and IP Tables packet marking?

Hi Dave, thanks for asking! I still use HFSC. The scheme works well if you are doing custom DCSP and I really like the real-time behavior. My current HFSC setup uses 2 real-time queues and 4 link share queues. the top real-time queue is for VOIP phones. The VOIP queue long term behavior is to have enough bandwidth for about 10 simultaneous calls... But for a burst of up to 20ms it gets 50 times that bandwidth. This makes my calls ROCK solid.

Next comes about 10Mbps of gaming traffic, which works great too, it bursts at 20x that rate for up to 20ms. Most of the time it's not a major player on my network but it's nice that my kids don't complain about lag.

Essentially all the real time traffic is UDP, so I don't use fq_codel below these classes, I use pfifo. The design of these queues is that they essentially never have more than a few packets in them due to their allowed burst rate.

My link share queues are kind of a Video, Normal, NFS/Samba, and Bulk system.

Video gets low latency through high bandwidth burst at 60% of the link, but long term is limited to something like 20% of the link. it's for both streaming and vidconf at the moment.

normal is where most stuff starts. It gets 20% of the link in burst, and 50% of the link in steady state. Stuff that transfers more than a few seconds of data continuously gets demoted to bulk.

The NFS queue is where all my network mounts sit. It gets 15% burst (so it is a time donor) and 20% long term.

Bulk gets 5% burst (it's a HUGE time donor) and 10% normally, with an overall hard limit of 95% of link (when its the only thing going at the moment, it still leaves a little room for new connections to initiate).

In addition to queues on the router, I use smart switches that have WRR behavior based on DSCP. This keeps purely local LAN traffic from interfering with high priority packets. This is an issues since multiple machines network mount home dirs via NFS4 and the NFS server has a bonded link and can saturate some of my wires and all of my APs. I also have one portion of my house served by a powerline adapter, which is limited to about 40Mbps in my switches. that limit plus WRR behavior results in a stable link across this bottleneck.

I think cake made some mistakes in it's DSCP mappings for diffserv4 that mismatch both with commonly available switches and with default Linux driver WMM mappings. but for the most part, cake is great. most people should try piece of cake, and then cake diffserv4 before anything else.

Why do I stick to HFSC? I really like the control of latency and bandwidth and the ability to separate real time from link share. I like being able to stick my high priority UDP traffic in something other than codel as it really shouldn't ever drop. (I probably should use RED in my UDP flows... but in truth it's never needed). I don't want dropped packets in my high priority UDP flows. I like the fact that the HFSC does a really nice packet-by-packet job of interleaving packets. It rarely if ever actually drops packets:

my lan egress queue:

qdisc hfsc 1: root refcnt 2 default 40 
 Sent 302603740730 bytes 456270028 pkt (dropped 11, overlimits 7353071 requeues 0) 
 backlog 0b 0p requeues 0

eleven packet dropped out of 456M

and my wan egress queue:

qdisc hfsc 1: root refcnt 2 default 40 
 Sent 88526041589 bytes 309218709 pkt (dropped 1, overlimits 1528385 requeues 0) 
 backlog 0b 0p requeues 0

It dropped ONE packet out of 309 million

What HFSC is really good at, when its class hierarchy is designed well, is interleaving packets in such a way that they all meet their time deadlines

of course it doesn't hurt that I have ATT gigabit GPON fiber :wink:

The home network environment, particularly for people with say home+office, has gotten rather closer to what used to be "enterprise" requirements. In say 2005-2010 entire college campuses of 10000 people had less bandwidth to the internet than I do to my home. For me it's super important that my family NEVER EVER garbles my business phone calls for example. Now that we're in coronavirus lockdown, it's also really important that my family NEVER EVER garbles my wife's video lectures or seminars. Having different priority items go into different queues designed for their purpose makes sense to me. Of course, we're running more like a business in my house than a typical home network.

Also, I'm using nftables for all my firewalls, and sophisticated DSCP marking schemes are quite easy in that system.

I had hoped to expand on that more, but OpenWrt seems to have issues with nftables in recent releases so I was stymied a bit. After a future release, where nftables works correctly, I will return to that to try to teach people how to DSCP mark their important traffic.

1 Like