SQM with two VPN

This is a continuation of this topic

Hello Community! I continue to build a network of my dreams! :smiley:

I want to make SQM work with two VPN. What I've done:
Two wireguard VPN
The first for traffic within my country
Second for external traffic
I did it with vpn-policy-routing, adding the ip subnets belonging to my country
What I want to do:
Make SQM works correctly with two VPN

I am REALLY still happy that @Lynx helped me make SQM work with one VPN, but I realized that I needed two VPN.

I think you can do this with this:

I sat, read, but did not understand which lines I need to change. Could you help me please? To be honest, I don't understand at all :frowning:

You might not need to change very much.

You would just have your VPN's both apply fwmark 4 (can be set in LuCi). If you don't have br-guest then you can delete the corresponding lines. Then copy in the nftables file, chmod +x it, then restart firewall. Then copy in the service file and hotplug file, chmod +x it, then start the service. Then:

tcpdump -i ifb-ul -v


tcpdump -i ifb-dl -v

to verify it's capturing the correct flows with appropriate DSCPs.

I guess this is a fairly advanced arrangement so you probably ought to try to take the time to learn the principles if you want to apply it to make sure you are doing the right thing. There's not much fun in applying a technique you don't understand.

1 Like

Okay, i will try it. Thank you for your help! Where i can set fwmark 4? I searched in internet, searched in luci by clicking to every page and still cant find it

1 Like

Oh my god! I'm really blind! I was looking in wrong place. Sorry!

So my actions are:
Set in both VPNs fw mark 0x4
then in cake-dual-ifb delete all lines that contain br-guest (i have many interfaces (for example guest interface, firstroom, secondroom and etc) , but i think it's not matter in my case?)
then in cake-dual-ifb change 50 and 51 lines with 80-90% of my correct speed (185mbit for dl/ul)
then in cake-dual-ifb.nft change lines or fully remove to exclude br-guest from everywhere
Then follow instruction
To install:

place cake-dual-ifb in /etc/init.d
chmod +x cake-dual-ifb
place 11-cake-dual-ifb in /etc/hotplug.d/iface/
chmod +x 11-cake-dual-ifb
place cake-dual-ifb.nft in /usr/share/nftables.d/ruleset-post/
edit cake-dual-ifb.nft for your use case
verify interfaces (e.g. replace or delete br-lan / br-guest lines as required)

Am i right or missing something?

Hmm well I think ideally you likely want to capture all unencrypted upload and download flows to and from the internet. That was my intention in designing this solution. If you don't capture all the flows then given the bypassing flows that cake doesn't see there could still be saturation of the connection beyond what is set in cake thereby leading to bufferbloat and those bypassing flows wouldn't be subject to per internal host IP fairness.

Perhaps you should draw a diagram showing all the interfaces and flows in your network?

1 Like

I'm not master of GIMP and I've never done this, but i tried : D

Here's what i got:

p.s. all switches/wifi APs works on OpenWRT, yeah i'm like this freedom what Owrt gives me : D

1 Like

Oh man that looks pretty complicated. So basically what I did with br-guest you'd have to do with all of those br-lan.XX interfaces. The recipe is there.

Bear in mind it's late and I'm tired so take the below with a pinch of salt. But I think something along the lines of the below is apt in your case.

So I would have nftables mark 1, 2, 3, ..., X-1. for each of those interfaces (just like I did for br-lan and br-guest) and then have OpenWrt mark traffic as X, and VPN1 and VPN2 mark X+1. And then you have to tc filter on those appropriate marks in cake-dual-ifb to collect the appropriate ingress and egress flows.

In code you have only br-lan and br-guest, my interfaces have br-lan.xx

So i should do something like that for all my br-lan.xx?

1 Like

Yes. It's exactly the same principle and will definitely work in your case. You just need to take care to mark in the manner I suggested above and then tc filter appropriately using marks to collect the flows.

You may not need to actually mark the different br-lan interfaces differently. I just did that to help separate things out in my mind.

care to mark in the manner I suggested above and then tc filter appropriately to collect the flows.

I don't think i understand.

So okay, i added/edited all lines in both files, then assigned to both VPNs FW mark 0x4

What else should i do?

I think you really need to try to figure out why I'm doing what I'm doing in this script so that you can apply to your own situation.

I will try, but for me there are sooo many things that are incomprehensible, and there is not much information on the Internet for this, it will be difficult for me to understand everything at once.

Well, am i right that i should add/edit lines in both files, assign to both VPNs fw mark 0x4 or im missing something?

What's incomprehensible? I think the best way forward is for you to figure out what each aspect of the firewall nftables script and the main init.d script actually do. Then you will be able to make this work in your setup and also handle any future changes too.

Nftables is pretty easy to read. It matches on packets and then applies actions on those matched packets.

The init.d script also matches on packets and applies actions on those packets.

The ultimate goal is to create the intermediate frame buffers (ifb-dl and ifb-ul) in which all the dl and ul flows have been combined together for cake to be applied on them.

Even if you don't want to try to understand what the scripts do then like I stated above you can just follow the same recipe used for br-lan and br-guest.

1 Like

I am very grateful to you for your help! Thank you!

Let me know if you have any problems understanding any aspect.

Looking at the code I really think it's pretty simple.

Lines like:

iifname { br-lan, br-guest } oifname { wan, vpn } goto process-lan-to-wan

would become:

iifname { br-lan.99, br-lan.11, ... } oifname { wan, vpn } goto process-lan-to-wan

And lines like:

		oifname br-lan mark set 1
		oifname br-guest mark set 2

would become:

		oifname br-lan.99 mark set 1
		oifname br-lan.11 mark set 2


Basically you just assign an fwmark for each interface in nftables and then you filter on those marks in the init.d script:

        # capture wan -> br-lan (egress) and restore DSCPs from conntracks
	# filter by fwmark 1
	tc filter add dev br-lan.XX parent 1: protocol ip handle 1 fw flowid 1:1 action ctinfo dscp 63 128 action mirred egress redirect dev ifb-dl

	# capture wan -> br-guest (egress) and restore DSCPs from conntracks
	# filter by fwmark 2
	tc filter add dev br-lan.YY parent 1: protocol ip handle 2 fw flowid 1:1 action ctinfo dscp 63 128 action mirred egress redirect dev ifb-dl

But like I wrote above I'd encourage you to try to work out what's going on with these two scripts because then it will be easy for you to apply to your own situation and most importantly test it to make sure it works. And also adapt it to change in the future.

This isn't designed so much as a user friendly 'plug and play' option. It's more a recipe that can easily be adapted to different scenarios and use cases.

@moeller0 do you think I'm right that it wouldn't be the right approach for me to try to make code that takes in parameters to try to do the right thing in different cases because everyone's network is so unique? For example does everyone always use bridge interfaces? I'm not sure. If everyone uses bridge interfaces then perhaps I could have the nftables script and the init.d script take in a list of interfaces and then work through each one. But I'm wary about such automation because this tool can easily bust things up so it's maybe bad to make it more user friendly. Maybe it just has to stay as a template. This template will work with little or no change in many configurations since mostly users just have br-lan and maybe also br-guest.

Anyway I'd be interested in your thoughts here.

1 Like

Well, try it! It might turn out not to be well-generalizable but without trying we will not know.

Again the best way to figure out is to try t generalize it and try to recruit other users with slightly different configs.

My gut feeling is the core idea should be generalizable. And even if not the work figuring this out is IMHO likely to improve your template (if only because you can add more informed content about places where things can get tricky).

But I have little first hand experience here to draw from :wink:

1 Like

OK @Openwrtfunboy I will try to generalise this out so you just put in your interfaces and it will work. It may require a few posts on this thread and some tweaking but we will get there. I hope to get to this over the course of the next few days.

1 Like