Policy Bases Routing (pbr) not working when mwan3 installed

I'm using the lates version of pbr (0.9.9-9) on openwrt 22.03.2 and this works like expected. Because off my Raspberry Pi 4b setup, with managed switches I only using the one network connection on my Pi.

for this reason the Pi doen't see if the wan interface is down. mwan3 can be configurated so it can detect a 'disconnect' I my case this is handy, because than it switches over to the backup wwan.

When I manual stop the wan service (without mwan3 installed) pbr detect this correctly and routes over the backup wwan. So this works correctly.

But when I install mwan3 all is routed through my wan, instead of the vpn for different clients. When I uninstall mwan3 all pbr routes are working as configured.

So also this threat, but there is not really an answer.
https://forum.openwrt.org/t/vpn-policy-based-routing-web-ui-discussion/10389/1198

I changed the following line in the pbr config;

option rule_create_option 'add'

in

option rule_create_option 'insert'  

My /etc/config/pbr

config policy
	option name 'ignore local traffic'
	option src_addr '10.2.0.0/24'
	option interface 'ignore'

config policy
	option name 'google'
	option dest_addr 'google.nl google.com'
	option interface 'ignore'

config policy
	option name 'Macbook Air Raymond'
	option src_addr '192.168.1.129'
	option interface 'wan'
	option enabled '0'

config policy
	option name 'Alle dhcp clients'
	option src_addr '192.168.1.100/30 192.168.1.104/29 192.168.1.112/28 192.168.1.128/25'
	option interface 'wg0'

config policy
	option name 'vpn server clients'
	option src_addr '192.168.200.2/31 192.168.200.4/30 192.168.200.8/29 192.168.200.16/28 192.168.200.32/28 192.168.200.48/31 192.168.200.50/32'
	option interface 'wg0'

config pbr 'config'
	option verbosity '2'
	option ipv6_enabled '0'
	list ignored_interface 'vpnserver'
	list ignored_interface 'wgserver'
	option boot_timeout '30'
	option rule_create_option 'insert'
	option procd_reload_delay '1'
	option webui_enable_column '1'
	option webui_protocol_column '1'
	option webui_chain_column '1'
	option webui_show_ignore_target '1'
	option webui_sorting '1'
	list webui_supported_protocol 'tcp'
	list webui_supported_protocol 'udp'
	list webui_supported_protocol 'tcp udp'
	list webui_supported_protocol 'icmp'
	list webui_supported_protocol 'all'
	option strict_enforcement '0'
	option resolver_set 'dnsmasq.ipset'
	option enabled '1'

config include
	option path '/usr/share/pbr/pbr.user.aws'

config include
	option path '/usr/share/pbr/pbr.user.netflix'
	option enabled '1'

My //etc/config/mwan3

config globals 'globals'
	option mmx_mask '0x3F00'

config interface 'wan'
	option enabled '1'
	option family 'ipv4'
	option initial_state 'online'
	option track_method 'ping'
	option count '1'
	option size '56'
	option max_ttl '60'
	option timeout '4'
	option interval '10'
	option failure_interval '5'
	option recovery_interval '5'
	option down '5'
	option up '5'
	option reliability '1'
	option check_quality '0'
	list track_ip '84.200.69.80'
	list track_ip '84.200.70.40'

config member 'wan_m1_w3'
	option interface 'wan'
	option metric '1'
	option weight '1'

config policy 'wan_only'
	list use_member 'wan_m1_w3'
	list use_member 'wwan_m1_w3'
	option last_resort 'unreachable'

config rule 'https'
	option sticky '1'
	option dest_port '443'
	option proto 'tcp'
	option use_policy 'wan_only'

config rule 'default_rule_v4'
	option dest_ip '0.0.0.0/0'
	option family 'ipv4'
	option proto 'all'
	option sticky '0'
	option use_policy 'wan_only'

config rule 'default_rule_v6'
	option dest_ip '::/0'
	option family 'ipv6'
	option proto 'all'
	option sticky '0'
	option use_policy 'wan_only'

config interface 'wwan'
	option enabled '1'
	option initial_state 'online'
	option family 'ipv4'
	option track_method 'ping'
	option count '1'
	option size '56'
	option max_ttl '60'
	option timeout '4'
	option interval '10'
	option failure_interval '5'
	option recovery_interval '5'
	option down '5'
	option up '5'
	list track_ip '84.200.69.80'
	list track_ip '84.200.70.40'
	option reliability '1'
	option check_quality '0'

config member 'wwan_m1_w3'
	option interface 'wwan'
	option metric '2'
	option weight '2'


My /etc/mwan3.user file (so it detects the wan interface 'disconnect' tries to ifup again, this will not work and than mwan3 switched to my backup wwan)

#!/bin/sh

if [ "${ACTION}" = "disconnected" ] && [ "${INTERFACE}" = "wan" ] ; then
    ifup wan
fi

These packages have overlapping purposes and therefore a conflict would be expected. My recommendation is to uninstall pbr and try to use mwan3 for all your policy-based routing needs, even though it is cumbersome.

1 Like

That could be a solution in a way..

Another thought.. Can I use/make some script that checks if my wan interface is disconnected and if so, do the script I use in mwan3?

Because that is the only one problem I encounter at the moment. So if my router knows that the wan is disconnected it's all ok.

Can I not just use the script that I use for mwan3?

Other users reported successful co-existence of vpn-policy-routing and mwan3 on 21.02, so maybe roll back to that and try?

If you're handy with shell scripting, you might be able to achieve the interface-management functionality of mwan3 with the interface hotplug scripts and uninstall mwan3 then, even tho it's cumbersome.

¯\_(ツ)_/¯

PS. Might need to try older versions of vpn-policy-routing which used less efficient marking/matching code than the most recent one, but may have been more compatible with mwan3.

1 Like

Use of mwan3 with vpn-policy-routing was my old working config. So that is not an issue. I just upgraded to pbr because it's getting obsolete in the future. (at least thats in your readme somewhere :wink:

But it is an idea offcourse for now.

Until mwan3 is rewritten in nft, I don't think it makes sense for me or @feckert to spend time investigating compatibility on 22.03 with iptables version of both packages.

3 Likes