Ultimate SQM settings: Layer_cake + DSCP marks

Yes, That's exactly what I want.

Thanks for your valuable suggestions. I will definitely try that approach if not now.

i think this what you need: https://superuser.com/questions/764986/howto-setup-a-veth-virtual-network/765078#765078

It's an interesting idea, and it might work, it's certainly easier to try than the namespace idea. I have an old tp-link router around I could maybe test it on but it will be many days before I can get to that. If someone else can try it, please report back here.

@dlakelan
i will try when network is not in use!
i think there's one more solution, which is creating a tuntap , then route traffic between wan and lan.
but not sure how!

That is undoubtedly going to be much much higher CPU requirement, as it will send each packet one at a time from the kernel to userspace and back to the kernel.

I set a test up on an old TP-link WDR-3600 I have lying around.
with ip-full and kmod-veth and all that installed, I used this script to create veth pair:

ip link add type veth
ip link set veth0 up
ip link set veth1 up

I then went to luci and created a new interface "prewan", I made it a bridge between eth0.2 and veth0, then I moved WAN to be physically connected to veth1.

I then put SQM instance on egress only on veth0 and veth1. The veth0 rate was "download" rate of actual wan, and the veth1 rate was "upload" rate of actual wan.

and IT WORKED just fine.

EDIT: obviously you want to fiddle around with the iptables rules for tagging with DSCP, and perhaps it requires you to do use bridge netfilter to ensure the iptables are called when the prewan bridge receives packets. But it's a very cool idea. @moeller0 might be interested.

So I did wind up playing with this on my main router, and found that I had to shove the bridge into a different namespace otherwise it wouldn't forward the packets through the veth device on receive, it would just receive them on the bridge itself. But after shoving the bridge into the other namespace where none of the ip addresses existed it would just forward from the wan ethernet to the veth where it queues and then pops out on veth1 in the main namespace. Definitely the way to go to get fully customizable dscp based qos/sqm

Just a thought: for most users the level of indirection/raffinesse that @hisham2630 needs seems a bit over the top, since most ISP, I know of, will not offer different rates for different traffic anyways, but just use a single shaper/policer per line. And in that case I begin to wonder whether it would not be simpler to recommend people desiring ultimate control to use a router+AP setup, then the ingress shaper can easily be instantiated on the LAN-facing interface and iptables marking will simply work without the need for either an IFB or the nifty veth-pair approach so skillfully demonstrated here.
Sidequestion @dlakelan can you quantify the CPU-cost of the veth approach as compared to the IFB approach? I ask, as maybe adding a shaper script that uses the veth-method to sqm-scripts would offer a partial solution for people wanting to play with setting the dscps for ingress without forcing them to use tc directly (which supposedly is computationallt cheaper than iptables and might still be a reasonable idea if the rules do not require access the the NATed internal addresses)?

I still have putting the newish nft-qos package through the paces to see whether that offers fine-grained shaping without introducing severe bufferbloat by itself.
Why do I bring this up here? Well, the intention for sqm-scripts was always to find a solution that works well enough for most users with needing to much individual twaeking and tuning. And I do consider the gaming case to be within reach (again @hisham2630's situation really requires the sophistication demonstrated here, but I bet a number of other's might be fine without the extreme heuristics involved in per-port rules).

IMHO this router & AP approach alao will scale better to higher speeds as it relieves the router from having to spend a variable amount of CPU cycles on WIFI. In addition this also removes the requirement for the router to offer WIFI itself, making a number of x86/arm boards a more natural fit.

Hishams case is definitely unusual but it's not unusual to have a lan and guest and even maybe a third vlan. In those cases it's problematic to do wan and lan shapers since there are two or three lans and you can't prioritize them as @dsinghra12 wants to.

I will do some more fiddling as my setup isn't yet stable, I need to put my dscp rules into the separate namespace etc.

I can say that strangely my network seems to be faster after the veth stuff, though I suspect it's due to the variability of my wan speed. Gigabit fiber from ATT isn't really gigabit, as the whole neighborhood probably shares a single 10gig uplink or something, result is speeds vary from 400mbit to 900

@dlakelan I will try above mentioned approaches by tomorrow as I have only one router with me now and need to give some Demo on that today. Will update my results by tomorrow.

Hi, in Iraq(which is my country),Egypt and India, they offer different traffic rates, like 10 mbps for normal download and 50 mbps for google drive, even youtube will have something like 20mbps.

i see that cpu cost using veth method will be + 10~15% more than normal using 2 pair's of veth.
it's much easier to use iptables cause it's more readable and understandable than tc commands.also tc is
well known to be not fully documented!, i know that tc is cheaper but hard to configure!

i was planing to add a new tab in sqm, lets call it advanced, in this tab user can easily select which ip's/port's
he want to tag and prioritize. i'm still fine without using the second veth pair, i'm still get a stable ping with
saturated link in both or one direction(not a 100% saturated lets say 98% or less).
i'm still want to buy a x86/x64 router, but still don't know which brands and models is available in my country.

i'm still want to understand/get this case to work, also preferably without using second namespace!(try to add an ip for wan bridge and see if this help!), you should be happy if you can get 400mbps, here you can't even get a 8mbps!. in the north of the country
you can get 1gbps fo 100$ but not sure if it's unlimited and stable!
EDIT:
It would be much better if there's a way to put one veth(or use another type of interface) in main table, and remove wan from main table, and second table with another veth with wan interface!

IMHO something like this should live in its own independent luci-app, call it luci-app-diffserve or so. This kind of fine-grained configuration is pretty much what sqm-scripts was created to avoid (for those user's that can live without extreme configurability).
As I already indicated adding a script to sqm-scripts that uses the veth-pair method instead of the ifb route should be easier to get upstreamed :wink:

this is another good idea,but who will write this new package!
@dlakelan there's an interesting things here
https://wiki.debian.org/BridgeNetworkConnections
and this
https://wiki.libvirt.org/page/Networking#Altering_the_interface_config
it might help,
under Libvirt and bridging.

I think what happens is that the Linux bridge sees a packet come in say eth0.2 into the bridge. It checks this packet to see how to deliver it. it sees that it's destined for the router and then it processes the packet rather than resending it out the veth0.

Now, for many people this may not be particularly a big problem, since they are mostly routing but I also run squid proxy on my router to fine-grain control access to the web (it's for security, and time-limits for kids, and also some prioritization/bandwidth limits as well). So this means all the traffic going to squid, which is a lot, is un-queued.

Moving the bridge to the second namespace solves that problem. But it may not be a problem for most people.

Try it like this from above and see if it works: Ultimate SQM settings: Layer_cake + DSCP marks - #251 by dlakelan

Hey @dlakelan
i think using ebtables commands will fix this problem!

It might, but it would also add a bunch of cpu overhead compared to just shoving it in a separate namespace I believe.

i have another idea but not sure if it's right or will work properly, the idea is:
1.create a new bridge lets call it br-all.
2.add other br-vlan's to this br-all.
3. use ebtables to prevent each br-vlans to not talk to each other.
4.add veth1 to br-all.
5.add veth0 to a new routing table, then use sqm on veth0.
can you kindly post the commands that you have used to create the second name space?!

also I think this could work, but becomes much more of a security issue, and also will kinda break the structure of the OpenWrt firewall I think.

For most people, I think just create the bridge between the wan ethernet and veth0 and off you go, no namespace needed and no strange ebtables manipulation unless a lot of services like squid or something run on the router, in which case, just shove the prewan into a separate namespace and it works. It's not more than a few commands to do the namespace thing, like 5 or 6, to set up the namespace, to move the devices, and then create the bridge in the other netns and load the DSCP tagging rules.

so if someday i created a guest network, then is it enough to just bridge between the wan ethernet and veth0.
then establish a wan connection on veth1.
after this i have to use sqm on veth0 and veth1 ? is this correct!
EDIT:
do you bridge the second namespace to the original machine default space.