According to https://support.google.com/youtube/answer/2853702?hl=en#zippy=%2Cp-bei-fps%2Cp-fps 1440@60 takes * Video Bitrate Range: 9,000–18,000 Kbps or
(9000 * 1000)/(8 * 1024^2) = 1.07 to (18000 * 1000)/(8 * 1024^2) = 2.15 MB/sec.
But again that is not smoothly paced out but rushed as fast as possible, and that can result in your shaper cutting in reducing those bursts resulting in retransmissions to fill the gaps and the sender slowing down.
that might happen as well, I will keep an eye for it and thanks for the explanations.
I'm trying to adapt this to OpenMPTCPRouter using two wans bonded... i'm unsure if i should create 2 more veth interfaces or add the second wan to the veth0 and veth1 tables?
also OMR uses iptables firewall package and this script uses nft. is there an issue running both?
i've gone with the first approach and it seems the bulk isn't hitting my newly created veths. perhaps you could guide me in the right direction to accomplish this? it's the same idea, just with 2 wans.
You can just create IFBs that combine all the flows as needed. See here for inspiration:
I have configured router 1 in bridge mode and router 2 OpenWrt in pppoe . I modified the script by replacing wan with pppoe-wan, the rules and DSCP are in place in the firewall, unfortunately I lose the internet connection. I have to delete the script in root to get internet back. what's going on? Any ideas?
Did the firewall file load correctly? If you issue firewall restart:
/etc/init.d/firewall restart
do you see any errors? Or you can run:
nft -c -f /use/share/nftables.d/ruleset-post/cake-dual-ifb.nft
to check that for errors.
In earlier 22.03 snapshots if the firewall include has any errors it will default to a completely empty ruleset and result in loss of internet connectivity. This behaviour has been changed in more recent 22.03 snapshots after I raised the issue.
At the absolute minimum you have to delete the XX, YY lines in the bridge table where I put IoT MAC addresses. I set all those to dscp for bulk traffic. Thinking about it I should comment that rule.
Also check for any errors from the tc calls by looking at the log file in: /tmp/cake-dual-ifb.log.
Ok thanks a lot, I'll check it out!
I have read through both the entire Qosify and this thread, but I am still confused.
Could someone please explain what the differences are between this script and Qosify?
When would you use one over the other?
This one seems like a more complete “batteries included” solution.
Yet qosify seems much more popular. Does it have any technical advantages over this script?
Would it make sense to adapt the rules used in here to Qosify?
(I am still waiting for my R4S to arrive. So I can’t try both of them out and compare. I’m still in my research phase.)
Welcome to the world of QoS 
The biggest differences are, qosify employs eBPF and hence can change DSCP values before the "normal" ingress solution of traffic-shaper+AQM on an IFB (intermediate block device) hanging on the true WAN device sees packets; the script from this thread uses veth interfaces to route the ingress traffic through an veth pair so it can preform ingress shaping on a veth's egress side. The upshot of that configuration is that now nftables can be used to change DSCP values, so this script uses nftables and not eBPF code to change the ingress DSCP values. A consequence of that is that this script offers richer filter and selection methods than qosify for ingress, but not everybody really needs that.
IMHO the two things qosify could still learn is setting DSCPs based on local port numbers, based on local internal (inside NAT) addresses and on combinations of IP addresses and ports.
That said, I am pretty boring myself and my network just works well with just sqm/cake's normal behavior (using cake's per-internal-IP fairness mode) so that I do not bother to set up QoS rules and hence only comment from the peanut gallery here...
Which IMHO is not a "plus" the core idea of prioritization is that for everything treated to better service other things need to be treated to worse service, so prioritization requires a certain restraint.
My take home message is to only every up-prioritize the absolute minimum one can get away with and only after conforming that such a specific prioritization rue actually helps. So having an elaborate QoS hierarchy someone else built and then one does not fully understand seems a bit dangerous as it might lead to trying to prioritize too many packets.
Thank you for your super quick reply. Now everything makes a lot more sense.
Yes, I saw your warning repeatedly in the other thread and really appreciate it.
I have reviewed this script and understand most of the prioritizations. It makes a lot of sense to prioritize audio/video conferencing, games and video streaming while deprioritizing other downloads.
The area I am concerned with is that (if I understand the script correctly), it seems to wash all incoming traffic to bulk (cs1) and then uses other rules to either prioritize to default or higher.
Particularly, I am wondering how the reservation of bandwidth works.
For example, if there is only bulk traffic, will cake allow it to use the entire available bandwidth?
Or will it keep the bandwidth reserved for voice and video traffic even if there currently isn’t any?
But I think I just need to set it up and then connect Wireshark and monitor how different traffic gets classified.
Yeah, sorry I keep harping on this, I just think that this is important enough everybody trying anything QoS is aware of it.
So DSCPs carry no inherent meaning and typically are only meaningful inside each individual diffserv domain, so we are not guaranteed to get useful/meaningful DSCPs from the internet/our ISP so resetting all packets' DSCP field to something harmless is IMHO the right thing to do (there are potential exceptions but these are rare and require your upstream to give you some guarantees).
Wheth ethe default should be CS1/LE or CS0 can be argued about until the cows come home. IMHO this is more a matter of personal preference and equally valid.
Yeah, cake is different, it will only reserve bandwith for the bulk class and only something like a 16/th of capacity, for other traffic each class will be scheduled preferentially for a certain amount of traffic and above that it will be scheduled round-robin (IIRC) with the lower classes. The upshot is that one should avoid packing greedy capacity seeking traffic into the higher classes (not a strict rule, but CS0 is IMHO the most suitable home for capacity seeking traffic in cake).
Yes of course, priority classes/tins will only affect scheduling if more than one contains packets.
EDIT: You wrote Wireshark, which I read but thought wireguard, so ignore my response. About wireshark/packetcaptures, yes that can be helpful, but keep in mind that changed DSCPs will only be visible on certain capturing places, e.g. if you capture on the WAN interface ingress packets with this script will all show the original internet/ISP DSCP values. So also look at the output of tc -s qdisc to see whether your test traffic increased the packet or byte counters in the expected cake tin.
Keep in mind that to cake all wireshark encrypted traffic will look like a single flow and hence it will not get as much tcapacity share as you might desire. @Lynx has cone up with an ingenious way of combining IFBs with policy based routing IIRC that sends wireguard traffic though the shaper unencrypted. If that is your problem, talk to him you will not regret it
(I use neither elaborate QoS nor wireguard so can only argue on principles, @Lynx brings the goods as he actually has first hand experience).
He said “wireshark” not “Wireguard”. 
Arg, yes you are right, and I am apparently too tired distracted and should call it a day
No need to be sorry at all. I actually appreciate it, since it's easy to go totally crazy on this stuff.
And thank you for all the other contributions too. I have been reading a lot of threads cover to cover the paste few days and you are super helpful in the majority of them. Thank you for leaving a massive trail of knowledge behind 
Awesome, that makes me worry a lot less.
That's frustrating Is there a way to capture traffic in exactly the way that cake will see it? That would make debugging so much easier.
And finally one new question:
When using layer cake, does cake divide traffic into the tins by itself? For example, when I start a 10gb software update download.
Will cake by itself put it into the bulk download category or is that something that needs to be set up with rules?
Cake sorts packets into tins based purely on DSCPs so you need to make sure all packets carry a DSCP that steers them into the desired tin. This threads script is very much about how to accomplish that.
It is not as dire as I made it sound, however you might need to capture the traffic on two interfaces to see both download and upload DSCPs. In both cases capturing on the interface cake is instantiated on should do the trick.
the most important for traffic management are those who manage everything the way I would say qosify does,
you are free to use either the first or the second version,
# Unmarked TCP traffic
meta nfproto ipv4 tcp sport != { 80, 443, 8080, 1935-1936, 2396, 2935 } tcp dport != { 80, 443, 8080, 1935-1936, 2396, 2935 } meta mark != 75 meta length 0-575 ip dscp cs1 counter meta mark set 80 comment \"Packet mark for unmarked TCP traffic of packet lengths between 0 and 575 bytes\"
meta nfproto ipv4 meta l4proto tcp meta length 0-575 ct direction reply meta mark 80 counter ip dscp set af41 comment \"Prioritize ingress unmarked traffic of packet lengths between 0 and 575 bytes to AF41 \(TCP\)\"
meta nfproto ipv4 meta l4proto tcp meta length 0-77 ct direction original meta mark 80 counter ip dscp set cs0 comment \"Prioritize egress unmarked traffic of packet lengths between 0 and 77 bytes to CS0 \(TCP\)\"
meta nfproto ipv4 meta l4proto tcp meta length 77-575 limit rate 230/second ct direction original meta mark 80 counter ip dscp set af41 comment \"Prioritize egress unmarked traffic of packet lengths between 77 and 575 bytes to AF41 \(TCP\)\"
meta nfproto ipv4 meta l4proto tcp meta length 77-575 limit rate over 230/second ct direction original meta mark 80 counter ip dscp set cs0 comment \"Deprioritize egress unmarked traffic of packet lengths between 77 and 575 bytes that have more than 230 pps to CS0 \(TCP\)\"
# Unmarked UDP traffic (Some games also tend to use really tiny packets on upload side (same range as ACKs))
meta nfproto ipv4 udp sport != { 80, 443 } udp dport != { 80, 443 } meta mark != 75 meta length 0-1256 limit rate over 230/second burst 100 packets ip dscp cs1 counter meta mark set 85 comment \"Packet mark for unmarked UDP traffic of packet lengths between 0 and 1256 bytes that have more than 230 pps\"
meta nfproto ipv4 meta l4proto udp numgen random mod 1000 \< 5 meta mark 85 counter meta mark set 0 comment \""0.5%" probability of unmark a packet that go over 230 pps to be prioritized to CS4 \(UDP\)\"
meta nfproto ipv4 meta l4proto udp meta length 0-77 ct direction reply meta mark 85 counter ip dscp set af41 comment \"Prioritize ingress unmarked traffic of packet lengths between 0 and 77 bytes that have more than 230 pps to AF41 \(UDP\)\"
meta nfproto ipv4 meta l4proto udp meta length 0-77 ct direction original meta mark 85 counter ip dscp set cs0 comment \"Prioritize egress unmarked traffic of packet lengths between 0 and 77 bytes that have more than 230 pps to CS0 \(UDP\)\"
meta nfproto ipv4 udp sport != { 80, 443 } udp dport != { 80, 443 } meta mark != { 75, 85 } meta length 0-1256 ip dscp cs1 counter ip dscp set cs4 comment \"Prioritize unmarked traffic of packet lengths between 0 and 1256 bytes that have less than 230 pps to CS4 \(UDP\) - Gaming "&" VoIP\"
second version
# Unmarked TCP traffic
meta nfproto ipv4 tcp sport != { 80, 443, 8080, 1935-1936, 2396, 2935 } tcp dport != { 80, 443, 8080, 1935-1936, 2396, 2935 } meta mark != { 40, 41 } meta length 0-1256 limit rate over 200/second burst 100 packets ip dscp cs1 counter meta mark set 45 comment \"Packet mark for unmarked TCP traffic of packet lengths between 0 and 575 bytes\"
meta nfproto ipv4 meta l4proto tcp numgen random mod 1000 \< 5 meta mark 45 counter meta mark set 0 comment \"Prioritize ingress unmarked traffic of packet lengths between 0 and 575 bytes to AF41 \(TCP\)\"
meta nfproto ipv4 meta l4proto tcp meta length 0-77 ct direction reply meta mark 45 counter ip dscp set af41 comment \"Prioritize egress unmarked traffic of packet lengths between 0 and 77 bytes to CS0 \(TCP\)\"
meta nfproto ipv4 meta l4proto tcp meta length 0-77 ct direction original meta mark 45 counter ip dscp set cs0 comment \"Prioritize egress unmarked traffic of packet lengths between 77 and 575 bytes to AF41 \(TCP\)\"
meta nfproto ipv4 meta l4proto tcp meta length \> 77 meta mark 45 counter ip dscp set af41 comment \"Deprioritize egress unmarked traffic of packet lengths between 77 and 575 bytes that have more than 230 pps to CS0 \(TCP\)\"
meta nfproto ipv4 tcp sport != { 80, 443, 8080, 1935-1936, 2396, 2935 } tcp dport != { 80, 443, 8080, 1935-1936, 2396, 2935 } meta mark != { 40, 41, 45 } meta length 0-1256 ip dscp cs1 counter ip dscp set cs4 comment \"Deprioritize egress unmarked traffic of packet lengths between 77 and 575 bytes that have more than 230 pps to CS0 \(TCP\)\"
# Unmarked UDP traffic (Some games also tend to use really tiny packets on upload side (same range as ACKs))
meta nfproto ipv4 udp sport != { 80, 443 } udp dport != { 80, 443 } meta mark != { 42, 43 } meta length 0-1256 limit rate over 250/second burst 100 packets ip dscp cs1 counter meta mark set 50 comment \"Packet mark for unmarked UDP traffic of packet lengths between 0 and 1256 bytes that have more than 230 pps\"
meta nfproto ipv4 meta l4proto udp numgen random mod 1000 \< 5 meta mark 50 counter meta mark set 0 comment \"0.5% probability of unmark a packet that go over 230 pps to be prioritized to CS4 \(UDP\)\"
meta nfproto ipv4 meta l4proto udp meta length 0-77 ct direction reply meta mark 50 counter ip dscp set af41 comment \"Prioritize ingress unmarked traffic of packet lengths between 0 and 77 bytes that have more than 230 pps to AF41 \(UDP\)\"
meta nfproto ipv4 meta l4proto udp meta length 0-77 ct direction original meta mark 50 counter ip dscp set cs0 comment \"Prioritize egress unmarked traffic of packet lengths between 0 and 77 bytes that have more than 230 pps to CS0 \(UDP\)\"
meta nfproto ipv4 meta l4proto udp meta length \> 77 meta mark 50 counter ip dscp set af41 comment \"Packet mark for unmarked UDP traffic of packet lengths between 0 and 1256 bytes that have more than 230 pps\"
meta nfproto ipv4 udp sport != { 80, 443 } udp dport != { 80, 443 } meta mark != { 42, 43, 50 } meta length 0-1256 ip dscp cs1 counter ip dscp set cs4 comment \"Prioritize unmarked traffic of packet lengths between 0 and 1256 bytes that have less than 230 pps to CS4 \(UDP\) - Gaming "&" VoIP\"
im getting the below error when running the script -
In file included from /dev/stdin:20:2-33:
/etc/nftables.d/00-rules.nft:254:21-25: Error: syntax error, unexpected string, expecting comma or '}'
ip6 daddr { IPV6::254 } meta l4proto { tcp, udp } th sport != { 80, 443, 8080, 1935-1936, 2396, 2935 } th dport != { 80, 443, 8080, 1935-1936, 2396, 2935 } meta mark != { 70, 71, 72, 73 } counter ip6 dscp set cs5 comment "Game consoles to CS5 (TCP and UDP)"
^^^^^
In file included from /dev/stdin:20:2-33:
/etc/nftables.d/00-rules.nft:255:21-25: Error: syntax error, unexpected string, expecting comma or '}'
ip6 saddr { IPV6::254 } meta l4proto { tcp, udp } th sport != { 80, 443, 8080, 1935-1936, 2396, 2935 } th dport != { 80, 443, 8080, 1935-1936, 2396, 2935 } meta mark != { 70, 71, 72, 73 } counter ip6 dscp set cs5 comment "Game consoles to CS5 (TCP and UDP)"
^^^^^
any ideas?
just to add, after a reboot ive now no access to internet, unable to access any web pages, have done reset for now,
what is tour interface ? in router and script
wan eth1 ?
pppoe-wan in router and in script