Sorry, I'm not following that. Where are you referring to for switching interfaces?
Sorry, multitasking is not one of my virtues. I mean that if my memory serves me well with his solution you have two SQM instances, one in your WAN interface and another one in your LAN interfaces. With this approach you only need one in your WAN interface:
config queue 'eth0'
option debug_logging '0'
option verbosity '5'
option qdisc 'cake'
option linklayer 'ethernet'
option overhead '44'
option qdisc_advanced '1'
option qdisc_really_really_advanced '1'
option squash_dscp '0'
option squash_ingress '0'
option ingress_ecn 'ECN'
option egress_ecn 'ECN'
option download '90000'
option upload '35000'
option enabled '1'
option iqdisc_opts 'nat dual-dsthost ingress'
option script 'ctinfo_4layercake.qos'
option interface 'eth0'
option eqdisc_opts 'nat dual-srchost ack-filter docsis'
2 Likes
I didn't know that we can specify a script within the sqm config. Where is your ctinfo_4layercake.qos script located?
/usr/lib/sqm/ctinfo_4layercake.qos
1 Like
Ohhhh! How on earth did you figure that out? Or maybe I just missed that detail in a thread somewhere?
Regardless, I have switched back to only a WAN SQM queue and I'll see how it goes from here for a bit. Many thanks for pointing that out!
There's a functional description of what ctinfo_4layercake does SQM Logic approach for Internet Bandwith management as well as background in the script comments.
Part of the point on the act_ctinfo/savedscp pairing is to reduce the need for every packet to go through a 'complicated' set of iptables classification rules. We go through those rules once for the first egress packet, make a classification decision and set that decision in stone in the firewall connmark. act_ctinfo can then reference that decision (connmark) on both the ingress (ifb4interface) path AND egress (interface) path. There is no need for every packet to go through the classification decision process which is why you don't see your firewall counters increasing wildly in the marking chain/s.
This rule:
Chain POSTROUTING (policy ACCEPT 165M packets, 126G bytes)
pkts bytes target prot opt in out source destination
406K 106M QOS_CAKE_eth0 all -- any eth0 anywhere anywhere connmark match 0x0/0x1000000
acts as the gatekeeper, in essence if the 'DSCP Set' bit in connmark is false then we need to go and do some classification work, else if it is true then classification had been done and we just carry on (act_ctinfo will do the work for us)
As regards diffserv5 appearing as diffserv3, are you sure 'tc' and 'cake module' are both patched, installed, running. Due to the extra diffserv mode the netlink parameters passed between the two need to match. I've seen this once before but I had an unpatched 'tc' IIRC. I'm not sure what appetite there is upstream for 'yet another diffserv mode' but for me it solves a problem of 'lower priority than bulk' ie. where to put bittorrent 
5 Likes
This is tremendous, really. I very, very much appreciate the link you pointed me to, as well as the extra explanation here. There are a lot of SQM threads out there these days and it's hard to keep up with them all.
The logic you have here makes so much more sense and not having each packet traverse multiple rules repeatedly is resulting in a noticeable snappiness over previous SQM configurations I have tried. Your explanation to account for the discrepancy I was seeing between previous iptables counters and the counters with this method gives me confidence this is working as intended now.
For confirmation, though, this is intended to operate on a single SQM queue on the WAN interface, yes?
I'm relatively certain my cake module and tc are both patched. I know for certain the tc errors I was seeing about diffserv5 being unrecognized are now gone when I set diffserv5 in ctinfo_4layercake. I will go back to my build environment and double-check my patches and whatnot. Maybe I'll do a nuke (good 'ol distclean) and rebuild fresh with the patches again to see if I get a different result. I don't want to take up much of your time troubleshooting my issue, so I will try all the avenues I can before bugging you about the binaries side of this. 
I mean it when I say how appreciative I am that you and others are still working on improvements in this space. This is exciting stuff and my hope is to learn enough to start contributing in the future, as well.
1 Like
It operates exactly like layer_cake.sqm & piece_of_cake.sqm. ie. The single definition in /etc/config/sqm will use two instance of CAKE, one attached to the ingress path via an intermediate 'ifb' interface, the other attached to the egress path. So if the physical wan interface is eth0 you'll get ifb4eth0 (ingress) and eth0 (egress) interface/instances of cake.
1 Like
Well I've tried everything I can think to patch for diffserv5, but no dice yet... here's where I'm at:
root@OpenWrt:~# tc qdisc show dev eth0
qdisc cake 8018: root refcnt 5 bandwidth 24500Kbit diffserv5 dual-srchost nat nowash ack-filter split-gso rtt 100.0ms noatm overhead 18 mpu 64
qdisc ingress ffff: parent ffff:fff1 ----------------
root@OpenWrt:~# tc qdisc show dev ifb4eth0
qdisc cake 8019: root refcnt 2 bandwidth 450Mbit diffserv5 dual-dsthost nat nowash ingress no-ack-filter split-gso rtt 100.0ms noatm overhead 18 mpu 64
root@OpenWrt:~# tc -s -j qdisc show dev eth0
[{"kind":"cake","handle":"8018:","root":true,"refcnt":5,"options":{"bandwidth":3062500,"diffserv":"diffserv5","flowmode":"dual-srchost","nat":true,"wash":false,"ingress":false,"ack-filter":"enabled","split_gso":true,"rtt":100000,"raw":false,"atm":"noatm","overhead":18,"mpu":64,"fwmark":"0"},"bytes":654782540,"packets":466892,"drops":10734,"overlimits":1103761,"requeues":0,"backlog":127176,"qlen":84,"memory_used":913120,"memory_limit":4194304,"capacity_estimate":3062500,"min_network_size":29,"max_network_size":1500,"min_adj_size":64,"max_adj_size":1518,"avg_hdr_offset":14,"tins":[{"threshold_rate":191406,"sent_bytes":0,"backlog_bytes":0,"target_us":11864,"interval_us":106864,"peak_delay_us":0,"avg_delay_us":0,"base_delay_us":0,"sent_packets":0,"way_indirect_hits":0,"way_misses":0,"way_collisions":0,"drops":0,"ecn_mark":0,"ack_drops":0,"sparse_flows":0,"bulk_flows":0,"unresponsive_flows":0,"max_pkt_len":0,"flow_quantum":300},{"threshold_rate":3062500,"sent_bytes":657270400,"backlog_bytes":127176,"target_us":5000,"interval_us":100000,"peak_delay_us":21649,"avg_delay_us":9010,"base_delay_us":4069,"sent_packets":477700,"way_indirect_hits":2081,"way_misses":1425,"way_collisions":0,"drops":1123,"ecn_mark":0,"ack_drops":9611,"sparse_flows":2,"bulk_flows":1,"unresponsive_flows":0,"max_pkt_len":30280,"flow_quantum":747},{"threshold_rate":765625,"sent_bytes":540,"backlog_bytes":0,"target_us":5000,"interval_us":100000,"peak_delay_us":136,"avg_delay_us":2,"base_delay_us":2,"sent_packets":10,"way_indirect_hits":0,"way_misses":9,"way_collisions":0,"drops":0,"ecn_mark":0,"ack_drops":0,"sparse_flows":1,"bulk_flows":0,"unresponsive_flows":0,"max_pkt_len":54,"flow_quantum":300}]},{"kind":"ingress","handle":"ffff:","parent":"ffff:fff1","options":{},"bytes":61945967,"packets":422401,"drops":0,"overlimits":0,"requeues":0,"backlog":0,"qlen":0}]
root@OpenWrt:~# tc -s -j qdisc show dev ifb4eth0
[{"kind":"cake","handle":"8019:","root":true,"refcnt":2,"options":{"bandwidth":56250000,"diffserv":"diffserv5","flowmode":"dual-dsthost","nat":true,"wash":false,"ingress":true,"ack-filter":"disabled","split_gso":true,"rtt":100000,"raw":false,"atm":"noatm","overhead":18,"mpu":64,"fwmark":"0"},"bytes":68969085,"packets":430257,"drops":0,"overlimits":39766,"requeues":0,"backlog":0,"qlen":0,"memory_used":93728,"memory_limit":15503360,"capacity_estimate":56250000,"min_network_size":46,"max_network_size":1500,"min_adj_size":64,"max_adj_size":1518,"avg_hdr_offset":14,"tins":[{"threshold_rate":3515625,"sent_bytes":0,"backlog_bytes":0,"target_us":5000,"interval_us":100000,"peak_delay_us":0,"avg_delay_us":0,"base_delay_us":0,"sent_packets":0,"way_indirect_hits":0,"way_misses":0,"way_collisions":0,"drops":0,"ecn_mark":0,"ack_drops":0,"sparse_flows":0,"bulk_flows":0,"unresponsive_flows":0,"max_pkt_len":0,"flow_quantum":858},{"threshold_rate":56250000,"sent_bytes":62525775,"backlog_bytes":0,"target_us":5000,"interval_us":100000,"peak_delay_us":12,"avg_delay_us":2,"base_delay_us":0,"sent_packets":322874,"way_indirect_hits":534,"way_misses":1439,"way_collisions":0,"drops":0,"ecn_mark":0,"ack_drops":0,"sparse_flows":3,"bulk_flows":1,"unresponsive_flows":0,"max_pkt_len":22710,"flow_quantum":1514},{"threshold_rate":14062500,"sent_bytes":6443310,"backlog_bytes":0,"target_us":5000,"interval_us":100000,"peak_delay_us":4,"avg_delay_us":1,"base_delay_us":0,"sent_packets":107383,"way_indirect_hits":0,"way_misses":12,"way_collisions":0,"drops":0,"ecn_mark":0,"ack_drops":0,"sparse_flows":1,"bulk_flows":0,"unresponsive_flows":0,"max_pkt_len":90,"flow_quantum":1514}]}]
The tc output indicates diffserv5 but if you parse the json from the interfaces, there are only 3 tins each. I'll keep plugging away at it until it works.
Just pushed an update to the requirements and install script for netdata-chart-sqm. Many thanks to @ced2git for finding that miss for me!
1 Like
I'm sorry to ask but do we have to patch tc and cake modul in order to use your script or is it just necessary for diffserv5? I would be fine with diffserv4 as I'm just trying to make DSCP work on the ingress side for my gaming consoles/devices.
@_FailSafe
Did you managed to get @ldir's script to work?
No you do not need the tc and cake patch for the standard support cake modes (incl diffserv4)
Diffserv5 is a special case.
1 Like
I updated https://github.com/ldir-EDB0/sqm-scripts/tree/exp/src to include a ctinfo_4layercake & ctinfo_5layercake, the only difference being one uses 'diffserv4', the other 'diffserv5'
6 Likes
Something that I just ran into is the interface naming around lines 98 & 99:
network_get_subnet SUBNET lan
network_get_prefix6 PREFIX wan6
I realized this morning that $PREFIX was empty in my case and I am very much a dual-stack environment. I had to update the wan6 to WAN6 in my config to match the case of the interface name in /etc/config/network.
Just wanted to put that out as a heads-up for anyone else who is tinkering with this. 
I had a quick look at a default /etc/config/network and this says:
config zone
option name wan
list network 'wan'
list network 'wan6'
No idea if there's a better way to determine stuff like that.
1 Like
something like:
ubus list network.interface.* | sed -n -e 's/network\.interface\.\(.*\)/\1/p' | awk '!/loopback/'
helpful?
apologies if this is not relevant...
2 Likes
@ldir It looks like this might be a good fit for the IPv6 WAN interface, at least.
network_find_wan6 ifc_wan6
network_get_subnet SUBNET lan
network_get_prefix6 PREFIX "${ifc_wan6}"
3 Likes
Updated the script to include a slight variation of that - I realised I'd written something similar in the miniupnpd.hotplug script.
2 Likes
No, I’m on cable. Wish I could get fiber where I’m located 