I'm trying to force the output interface for two different wireguard links so that each one is connecting to my server through a different output interface.
I set up a rule in mwan3 based on UDP destination port and IP address, but its count stays at 0
I see that my destination server IP is set as a connection in 'mwan3 connected' so probably the issue is there.
Could you post the output of uci export mwan3; iptables-save -c (in preformatted text </>)
Is this related to your other threads?
package mwan3
config rule 'wg0'
option family 'ipv4'
option use_policy 'wan_only'
option sticky '0'
option dest_port '1194'
option proto 'udp'
option dest_ip 'vpn_server_ip'
config rule 'https'
option sticky '1'
option dest_port '443'
option proto 'tcp'
option use_policy 'balanced'
config rule 'default_rule_v4'
option dest_ip '0.0.0.0/0'
option use_policy 'balanced'
option family 'ipv4'
config rule 'default_rule_v6'
option dest_ip '::/0'
option use_policy 'balanced'
option family 'ipv6'
config globals 'globals'
option mmx_mask '0x3F00'
option rtmon_interval '5'
option logging '1'
option loglevel 'notice'
config interface 'wan'
list track_ip '8.8.4.4'
list track_ip '8.8.8.8'
list track_ip '208.67.222.222'
list track_ip '208.67.220.220'
option family 'ipv4'
option reliability '2'
option count '1'
option timeout '2'
option interval '5'
option down '3'
option up '8'
option initial_state 'online'
option track_method 'ping'
option size '56'
option max_ttl '60'
option check_quality '0'
option failure_interval '5'
option recovery_interval '5'
option enabled '1'
config interface 'wan6'
option enabled '0'
list track_ip '2001:4860:4860::8844'
list track_ip '2001:4860:4860::8888'
list track_ip '2620:0:ccd::2'
list track_ip '2620:0:ccc::2'
option family 'ipv6'
option reliability '2'
option count '1'
option timeout '2'
option interval '5'
option down '3'
option up '8'
config interface 'wanb'
list track_ip '8.8.4.4'
list track_ip '8.8.8.8'
list track_ip '208.67.222.222'
list track_ip '208.67.220.220'
option family 'ipv4'
option reliability '1'
option count '1'
option timeout '2'
option interval '5'
option down '3'
option up '8'
option enabled '1'
option initial_state 'online'
option track_method 'ping'
option size '56'
option max_ttl '60'
option check_quality '0'
option failure_interval '5'
option recovery_interval '5'
config interface 'wanb6'
option enabled '0'
list track_ip '2001:4860:4860::8844'
list track_ip '2001:4860:4860::8888'
list track_ip '2620:0:ccd::2'
list track_ip '2620:0:ccc::2'
option family 'ipv6'
option reliability '1'
option count '1'
option timeout '2'
option interval '5'
option down '3'
option up '8'
config member 'wan_m1_w3'
option interface 'wan'
option metric '1'
option weight '3'
config member 'wan_m2_w3'
option interface 'wan'
option metric '2'
option weight '3'
config member 'wanb_m1_w2'
option interface 'wanb'
option metric '1'
option weight '2'
config member 'wanb_m2_w2'
option interface 'wanb'
option metric '2'
option weight '2'
config member 'wan6_m1_w3'
option interface 'wan6'
option metric '1'
option weight '3'
config member 'wan6_m2_w3'
option interface 'wan6'
option metric '2'
option weight '3'
config member 'wanb6_m1_w2'
option interface 'wanb6'
option metric '1'
option weight '2'
config member 'wanb6_m2_w2'
option interface 'wanb6'
option metric '2'
option weight '2'
config policy 'wan_only'
list use_member 'wan_m1_w3'
list use_member 'wan6_m1_w3'
config policy 'wanb_only'
list use_member 'wanb_m1_w2'
list use_member 'wanb6_m1_w2'
config policy 'balanced'
list use_member 'wan_m1_w3'
list use_member 'wanb_m1_w2'
list use_member 'wan6_m1_w3'
list use_member 'wanb6_m1_w2'
config policy 'wan_wanb'
list use_member 'wan_m1_w3'
list use_member 'wanb_m2_w2'
list use_member 'wan6_m1_w3'
list use_member 'wanb6_m2_w2'
config policy 'wanb_wan'
list use_member 'wan_m2_w3'
list use_member 'wanb_m1_w2'
list use_member 'wan6_m2_w3'
list use_member 'wanb6_m1_w2'
# Generated by iptables-save v1.8.3 on Wed Mar 10 17:03:44 2021
*nat
:PREROUTING ACCEPT [109448:10850704]
:INPUT ACCEPT [51843:3314634]
:OUTPUT ACCEPT [21915:1719940]
:POSTROUTING ACCEPT [8529:582486]
:postrouting_IOT_rule - [0:0]
:postrouting_pvtlan_rule - [0:0]
:postrouting_rule - [0:0]
:postrouting_vpnMF_rule - [0:0]
:postrouting_wan_rule - [0:0]
:postrouting_wgMF_rule - [0:0]
:prerouting_IOT_rule - [0:0]
:prerouting_pvtlan_rule - [0:0]
:prerouting_rule - [0:0]
:prerouting_vpnMF_rule - [0:0]
:prerouting_wan_rule - [0:0]
:prerouting_wgMF_rule - [0:0]
:zone_IOT_postrouting - [0:0]
:zone_IOT_prerouting - [0:0]
:zone_pvtlan_postrouting - [0:0]
:zone_pvtlan_prerouting - [0:0]
:zone_vpnMF_postrouting - [0:0]
:zone_vpnMF_prerouting - [0:0]
:zone_wan_postrouting - [0:0]
:zone_wan_prerouting - [0:0]
:zone_wgMF_postrouting - [0:0]
:zone_wgMF_prerouting - [0:0]
[109448:10850704] -A PREROUTING -m comment --comment "!fw3: Custom prerouting rule chain" -j prerouting_rule
[448:53004] -A PREROUTING -i pppoe-wan -m comment --comment "!fw3" -j zone_wan_prerouting
[54:12771] -A PREROUTING -i eth0.5 -m comment --comment "!fw3" -j zone_wan_prerouting
[0:0] -A PREROUTING -i tun0 -m comment --comment "!fw3" -j zone_vpnMF_prerouting
[8181:1109187] -A PREROUTING -i br-IOT -m comment --comment "!fw3" -j zone_IOT_prerouting
[100588:9643036] -A PREROUTING -i br-pvtlan -m comment --comment "!fw3" -j zone_pvtlan_prerouting
[131:8090] -A PREROUTING -i wg0 -m comment --comment "!fw3" -j zone_wgMF_prerouting
[4:424] -A PREROUTING -i wg1 -m comment --comment "!fw3" -j zone_wgMF_prerouting
[0:0] -A PREROUTING -i bond0 -m comment --comment "!fw3" -j zone_wgMF_prerouting
[78582:9133835] -A POSTROUTING -m comment --comment "!fw3: Custom postrouting rule chain" -j postrouting_rule
[55246:5981307] -A POSTROUTING -o pppoe-wan -m comment --comment "!fw3" -j zone_wan_postrouting
[14807:2570042] -A POSTROUTING -o eth0.5 -m comment --comment "!fw3" -j zone_wan_postrouting
[0:0] -A POSTROUTING -o tun0 -m comment --comment "!fw3" -j zone_vpnMF_postrouting
[2187:149688] -A POSTROUTING -o br-IOT -m comment --comment "!fw3" -j zone_IOT_postrouting
[1616:99420] -A POSTROUTING -o br-pvtlan -m comment --comment "!fw3" -j zone_pvtlan_postrouting
[2:212] -A POSTROUTING -o wg0 -m comment --comment "!fw3" -j zone_wgMF_postrouting
[2:212] -A POSTROUTING -o wg1 -m comment --comment "!fw3" -j zone_wgMF_postrouting
[1:84] -A POSTROUTING -o bond0 -m comment --comment "!fw3" -j zone_wgMF_postrouting
[2187:149688] -A zone_IOT_postrouting -m comment --comment "!fw3: Custom IOT postrouting rule chain" -j postrouting_IOT_rule
[8181:1109187] -A zone_IOT_prerouting -m comment --comment "!fw3: Custom IOT prerouting rule chain" -j prerouting_IOT_rule
[1616:99420] -A zone_pvtlan_postrouting -m comment --comment "!fw3: Custom pvtlan postrouting rule chain" -j postrouting_pvtlan_rule
[1616:99420] -A zone_pvtlan_postrouting -m comment --comment "!fw3: Custom pvtlan postrouting rule chain" -j postrouting_pvtlan_rule
[100588:9643036] -A zone_pvtlan_prerouting -m comment --comment "!fw3: Custom pvtlan prerouting rule chain" -j prerouting_pvtlan_rule
[100588:9643036] -A zone_pvtlan_prerouting -m comment --comment "!fw3: Custom pvtlan prerouting rule chain" -j prerouting_pvtlan_rule
[0:0] -A zone_vpnMF_postrouting -m comment --comment "!fw3: Custom vpnMF postrouting rule chain" -j postrouting_vpnMF_rule
[0:0] -A zone_vpnMF_postrouting -m comment --comment "!fw3" -j MASQUERADE
[0:0] -A zone_vpnMF_prerouting -m comment --comment "!fw3: Custom vpnMF prerouting rule chain" -j prerouting_vpnMF_rule
[70053:8551349] -A zone_wan_postrouting -m comment --comment "!fw3: Custom wan postrouting rule chain" -j postrouting_wan_rule
[70053:8551349] -A zone_wan_postrouting -m comment --comment "!fw3" -j MASQUERADE
[502:65775] -A zone_wan_prerouting -m comment --comment "!fw3: Custom wan prerouting rule chain" -j prerouting_wan_rule
[5:508] -A zone_wgMF_postrouting -m comment --comment "!fw3: Custom wgMF postrouting rule chain" -j postrouting_wgMF_rule
[135:8514] -A zone_wgMF_prerouting -m comment --comment "!fw3: Custom wgMF prerouting rule chain" -j prerouting_wgMF_rule
COMMIT
# Completed on Wed Mar 10 17:03:44 2021
# Generated by iptables-save v1.8.3 on Wed Mar 10 17:03:44 2021
*raw
:PREROUTING ACCEPT [21278483:15213525739]
:OUTPUT ACCEPT [240037:70070629]
:zone_IOT_helper - [0:0]
:zone_pvtlan_helper - [0:0]
:zone_wgMF_helper - [0:0]
[6127354:9143355639] -A PREROUTING -i br-IOT -m comment --comment "!fw3: IOT CT helper assignment" -j zone_IOT_helper
[8015597:2455402224] -A PREROUTING -i br-pvtlan -m comment --comment "!fw3: pvtlan CT helper assignment" -j zone_pvtlan_helper
[25279:1547612] -A PREROUTING -i wg0 -m comment --comment "!fw3: wgMF CT helper assignment" -j zone_wgMF_helper
[4:424] -A PREROUTING -i wg1 -m comment --comment "!fw3: wgMF CT helper assignment" -j zone_wgMF_helper
[2:168] -A PREROUTING -i bond0 -m comment --comment "!fw3: wgMF CT helper assignment" -j zone_wgMF_helper
COMMIT
# Completed on Wed Mar 10 17:03:44 2021
# Generated by iptables-save v1.8.3 on Wed Mar 10 17:03:44 2021
*mangle
:PREROUTING ACCEPT [21278486:15213525895]
:INPUT ACCEPT [243098:27772991]
:FORWARD ACCEPT [21034979:15185720160]
:OUTPUT ACCEPT [240040:70073057]
:POSTROUTING ACCEPT [21274062:15255747808]
:mwan3_connected - [0:0]
:mwan3_hook - [0:0]
:mwan3_iface_in_wan - [0:0]
:mwan3_iface_in_wanb - [0:0]
:mwan3_ifaces_in - [0:0]
:mwan3_policy_balanced - [0:0]
:mwan3_policy_wan_only - [0:0]
:mwan3_policy_wan_wanb - [0:0]
:mwan3_policy_wanb_only - [0:0]
:mwan3_policy_wanb_wan - [0:0]
:mwan3_rule_https - [0:0]
:mwan3_rules - [0:0]
[22765737:16131474949] -A PREROUTING -j mwan3_hook
[35137:2106772] -A FORWARD -o pppoe-wan -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment --comment "!fw3: Zone wan MTU fixing" -j TCPMSS --clamp-mss-to-pmtu
[35096:2076776] -A FORWARD -i pppoe-wan -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment --comment "!fw3: Zone wan MTU fixing" -j TCPMSS --clamp-mss-to-pmtu
[1461:85808] -A FORWARD -o eth0.5 -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment --comment "!fw3: Zone wan MTU fixing" -j TCPMSS --clamp-mss-to-pmtu
[1491:86396] -A FORWARD -i eth0.5 -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment --comment "!fw3: Zone wan MTU fixing" -j TCPMSS --clamp-mss-to-pmtu
[0:0] -A FORWARD -o tun0 -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment --comment "!fw3: Zone vpnMF MTU fixing" -j TCPMSS --clamp-mss-to-pmtu
[0:0] -A FORWARD -i tun0 -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment --comment "!fw3: Zone vpnMF MTU fixing" -j TCPMSS --clamp-mss-to-pmtu
[253158:74107121] -A OUTPUT -j mwan3_hook
[8039972:4107435527] -A mwan3_connected -m set --match-set mwan3_connected dst -j MARK --set-xmark 0x3f00/0x3f00
[23018895:16205582070] -A mwan3_hook -j CONNMARK --restore-mark --nfmask 0x3f00 --ctmask 0x3f00
[137787:13083307] -A mwan3_hook -m mark --mark 0x0/0x3f00 -j mwan3_ifaces_in
[136740:12991050] -A mwan3_hook -m mark --mark 0x0/0x3f00 -j mwan3_connected
[72589:8831357] -A mwan3_hook -m mark --mark 0x0/0x3f00 -j mwan3_rules
[23018895:16205582070] -A mwan3_hook -j CONNMARK --save-mark --nfmask 0x3f00 --ctmask 0x3f00
[12348596:6560084844] -A mwan3_hook -m mark ! --mark 0x3f00/0x3f00 -j mwan3_connected
[0:0] -A mwan3_iface_in_wan -i pppoe-wan -m set --match-set mwan3_connected src -m mark --mark 0x0/0x3f00 -m comment --comment default -j MARK --set-xmark 0x3f00/0x3f00
[991:79307] -A mwan3_iface_in_wan -i pppoe-wan -m mark --mark 0x0/0x3f00 -m comment --comment wan -j MARK --set-xmark 0x100/0x3f00
[54:12771] -A mwan3_iface_in_wanb -i eth0.5 -m set --match-set mwan3_connected src -m mark --mark 0x0/0x3f00 -m comment --comment default -j MARK --set-xmark 0x3f00/0x3f00
[2:179] -A mwan3_iface_in_wanb -i eth0.5 -m mark --mark 0x0/0x3f00 -m comment --comment wanb -j MARK --set-xmark 0x300/0x3f00
[137781:13082549] -A mwan3_ifaces_in -m mark --mark 0x0/0x3f00 -j mwan3_iface_in_wan
[136725:12996975] -A mwan3_ifaces_in -m mark --mark 0x0/0x3f00 -j mwan3_iface_in_wanb
[14420:2605676] -A mwan3_policy_balanced -m mark --mark 0x0/0x3f00 -m statistic --mode random --probability 0.39999999991 -m comment --comment "wanb 2 5" -j MARK --set-xmark 0x300/0x3f00
[21453:3918584] -A mwan3_policy_balanced -m mark --mark 0x0/0x3f00 -m comment --comment "wan 3 3" -j MARK --set-xmark 0x100/0x3f00
[0:0] -A mwan3_policy_wan_only -m mark --mark 0x0/0x3f00 -m comment --comment "wan 3 3" -j MARK --set-xmark 0x100/0x3f00
[0:0] -A mwan3_policy_wan_wanb -m mark --mark 0x0/0x3f00 -m comment --comment "wan 3 3" -j MARK --set-xmark 0x100/0x3f00
[0:0] -A mwan3_policy_wanb_only -m mark --mark 0x0/0x3f00 -m comment --comment "wanb 2 2" -j MARK --set-xmark 0x300/0x3f00
[0:0] -A mwan3_policy_wanb_wan -m mark --mark 0x0/0x3f00 -m comment --comment "wanb 2 2" -j MARK --set-xmark 0x300/0x3f00
[36749:2305407] -A mwan3_rule_https -m mark --mark 0x0/0x3f00 -j MARK --set-xmark 0x300/0x3f00
[35776:2246180] -A mwan3_rule_https -m mark --mark 0x300/0x3f00 -m set ! --match-set mwan3_sticky_https src,src -j MARK --set-xmark 0x0/0x3f00
[35776:2246180] -A mwan3_rule_https -m mark --mark 0x0/0x3f00 -j MARK --set-xmark 0x100/0x3f00
[105:6975] -A mwan3_rule_https -m mark --mark 0x100/0x3f00 -m set ! --match-set mwan3_sticky_https src,src -j MARK --set-xmark 0x0/0x3f00
[105:6975] -A mwan3_rule_https -m mark --mark 0x0/0x3f00 -j mwan3_policy_balanced
[36749:2305407] -A mwan3_rule_https -m mark ! --mark 0xfc00/0xfc00 -j SET --del-set mwan3_sticky_https src,src
[36749:2305407] -A mwan3_rule_https -m mark ! --mark 0xfc00/0xfc00 -j SET --add-set mwan3_sticky_https src,src
[0:0] -A mwan3_rules -d vpn_server_ip/32 -p udp -m multiport --dports 1194 -m mark --mark 0x0/0x3f00 -j mwan3_policy_wan_only
[36749:2305407] -A mwan3_rules -p tcp -m multiport --dports 443 -m mark --mark 0x0/0x3f00 -j mwan3_rule_https
[35768:6517285] -A mwan3_rules -m mark --mark 0x0/0x3f00 -j mwan3_policy_balanced
COMMIT
# Completed on Wed Mar 10 17:03:44 2021
# Generated by iptables-save v1.8.3 on Wed Mar 10 17:03:44 2021
*filter
:INPUT DROP [360:168060]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
:forwarding_IOT_rule - [0:0]
:forwarding_pvtlan_rule - [0:0]
:forwarding_rule - [0:0]
:forwarding_vpnMF_rule - [0:0]
:forwarding_wan_rule - [0:0]
:forwarding_wgMF_rule - [0:0]
:input_IOT_rule - [0:0]
:input_pvtlan_rule - [0:0]
:input_rule - [0:0]
:input_vpnMF_rule - [0:0]
:input_wan_rule - [0:0]
:input_wgMF_rule - [0:0]
:output_IOT_rule - [0:0]
:output_pvtlan_rule - [0:0]
:output_rule - [0:0]
:output_vpnMF_rule - [0:0]
:output_wan_rule - [0:0]
:output_wgMF_rule - [0:0]
:reject - [0:0]
:syn_flood - [0:0]
:zone_IOT_dest_ACCEPT - [0:0]
:zone_IOT_forward - [0:0]
:zone_IOT_input - [0:0]
:zone_IOT_output - [0:0]
:zone_IOT_src_ACCEPT - [0:0]
:zone_pvtlan_dest_ACCEPT - [0:0]
:zone_pvtlan_forward - [0:0]
:zone_pvtlan_input - [0:0]
:zone_pvtlan_output - [0:0]
:zone_pvtlan_src_ACCEPT - [0:0]
:zone_vpnMF_dest_ACCEPT - [0:0]
:zone_vpnMF_forward - [0:0]
:zone_vpnMF_input - [0:0]
:zone_vpnMF_output - [0:0]
:zone_vpnMF_src_DROP - [0:0]
:zone_wan_dest_ACCEPT - [0:0]
:zone_wan_dest_REJECT - [0:0]
:zone_wan_forward - [0:0]
:zone_wan_input - [0:0]
:zone_wan_output - [0:0]
:zone_wan_src_REJECT - [0:0]
:zone_wgMF_dest_ACCEPT - [0:0]
:zone_wgMF_dest_REJECT - [0:0]
:zone_wgMF_forward - [0:0]
:zone_wgMF_input - [0:0]
:zone_wgMF_output - [0:0]
:zone_wgMF_src_ACCEPT - [0:0]
[10799:1375416] -A INPUT -i lo -m comment --comment "!fw3" -j ACCEPT
[232311:26398263] -A INPUT -m comment --comment "!fw3: Custom input rule chain" -j input_rule
[168400:20736549] -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -m comment --comment "!fw3" -j ACCEPT
[112:6604] -A INPUT -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -m comment --comment "!fw3" -j syn_flood
[929:73876] -A INPUT -i pppoe-wan -m comment --comment "!fw3" -j zone_wan_input
[55:12811] -A INPUT -i eth0.5 -m comment --comment "!fw3" -j zone_wan_input
[0:0] -A INPUT -i tun0 -m comment --comment "!fw3" -j zone_vpnMF_input
[11305:2084307] -A INPUT -i br-IOT -m comment --comment "!fw3" -j zone_IOT_input
[51328:3325606] -A INPUT -i br-pvtlan -m comment --comment "!fw3" -j zone_pvtlan_input
[5:530] -A INPUT -i wg0 -m comment --comment "!fw3" -j zone_wgMF_input
[4:424] -A INPUT -i wg1 -m comment --comment "!fw3" -j zone_wgMF_input
[0:0] -A INPUT -i bond0 -m comment --comment "!fw3" -j zone_wgMF_input
[21034998:15185731439] -A FORWARD -m comment --comment "!fw3: Custom forwarding rule chain" -j forwarding_rule
[20961974:15176271566] -A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -m comment --comment "!fw3" -j ACCEPT
[0:0] -A FORWARD -i pppoe-wan -m comment --comment "!fw3" -j zone_wan_forward
[0:0] -A FORWARD -i eth0.5 -m comment --comment "!fw3" -j zone_wan_forward
[0:0] -A FORWARD -i tun0 -m comment --comment "!fw3" -j zone_vpnMF_forward
[8012:1002351] -A FORWARD -i br-IOT -m comment --comment "!fw3" -j zone_IOT_forward
[64884:8449842] -A FORWARD -i br-pvtlan -m comment --comment "!fw3" -j zone_pvtlan_forward
[128:7680] -A FORWARD -i wg0 -m comment --comment "!fw3" -j zone_wgMF_forward
[0:0] -A FORWARD -i wg1 -m comment --comment "!fw3" -j zone_wgMF_forward
[0:0] -A FORWARD -i bond0 -m comment --comment "!fw3" -j zone_wgMF_forward
[5:260] -A FORWARD -m comment --comment "!fw3" -j reject
[10799:1375416] -A OUTPUT -o lo -m comment --comment "!fw3" -j ACCEPT
[229255:68705265] -A OUTPUT -m comment --comment "!fw3: Custom output rule chain" -j output_rule
[210111:66786164] -A OUTPUT -m conntrack --ctstate RELATED,ESTABLISHED -m comment --comment "!fw3" -j ACCEPT
[7444:625368] -A OUTPUT -o pppoe-wan -m comment --comment "!fw3" -j zone_wan_output
[9656:721647] -A OUTPUT -o eth0.5 -m comment --comment "!fw3" -j zone_wan_output
[0:0] -A OUTPUT -o tun0 -m comment --comment "!fw3" -j zone_vpnMF_output
[1740:526352] -A OUTPUT -o br-IOT -m comment --comment "!fw3" -j zone_IOT_output
[299:45226] -A OUTPUT -o br-pvtlan -m comment --comment "!fw3" -j zone_pvtlan_output
[2:212] -A OUTPUT -o wg0 -m comment --comment "!fw3" -j zone_wgMF_output
[2:212] -A OUTPUT -o wg1 -m comment --comment "!fw3" -j zone_wgMF_output
[1:84] -A OUTPUT -o bond0 -m comment --comment "!fw3" -j zone_wgMF_output
[493:21829] -A reject -p tcp -m comment --comment "!fw3" -j REJECT --reject-with tcp-reset
[496:65118] -A reject -m comment --comment "!fw3" -j REJECT --reject-with icmp-port-unreachable
[112:6604] -A syn_flood -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -m limit --limit 25/sec --limit-burst 50 -m comment --comment "!fw3" -j RETURN
[0:0] -A syn_flood -m comment --comment "!fw3" -j DROP
[4037:664132] -A zone_IOT_dest_ACCEPT -o br-IOT -m comment --comment "!fw3" -j ACCEPT
[8012:1002351] -A zone_IOT_forward -m comment --comment "!fw3: Custom IOT forwarding rule chain" -j forwarding_IOT_rule
[8012:1002351] -A zone_IOT_forward -m comment --comment "!fw3: Zone IOT to wan forwarding policy" -j zone_wan_dest_ACCEPT
[1558:80424] -A zone_IOT_forward -m comment --comment "!fw3: Zone IOT to pvtlan forwarding policy" -j zone_pvtlan_dest_ACCEPT
[0:0] -A zone_IOT_forward -m conntrack --ctstate DNAT -m comment --comment "!fw3: Accept port forwards" -j ACCEPT
[0:0] -A zone_IOT_forward -m comment --comment "!fw3" -j zone_IOT_dest_ACCEPT
[11305:2084307] -A zone_IOT_input -m comment --comment "!fw3: Custom IOT input rule chain" -j input_IOT_rule
[0:0] -A zone_IOT_input -m conntrack --ctstate DNAT -m comment --comment "!fw3: Accept port redirections" -j ACCEPT
[11305:2084307] -A zone_IOT_input -m comment --comment "!fw3" -j zone_IOT_src_ACCEPT
[1740:526352] -A zone_IOT_output -m comment --comment "!fw3: Custom IOT output rule chain" -j output_IOT_rule
[1740:526352] -A zone_IOT_output -m comment --comment "!fw3" -j zone_IOT_dest_ACCEPT
[11305:2084307] -A zone_IOT_src_ACCEPT -i br-IOT -m conntrack --ctstate NEW,UNTRACKED -m comment --comment "!fw3" -j ACCEPT
[1857:125650] -A zone_pvtlan_dest_ACCEPT -o br-pvtlan -m comment --comment "!fw3" -j ACCEPT
[64884:8449842] -A zone_pvtlan_forward -m comment --comment "!fw3: Custom pvtlan forwarding rule chain" -j forwarding_pvtlan_rule
[64884:8449842] -A zone_pvtlan_forward -m comment --comment "!fw3: Custom pvtlan forwarding rule chain" -j forwarding_pvtlan_rule
[64884:8449842] -A zone_pvtlan_forward -m comment --comment "!fw3: Zone pvtlan to vpnMF forwarding policy" -j zone_vpnMF_dest_ACCEPT
[64884:8449842] -A zone_pvtlan_forward -m comment --comment "!fw3: Zone pvtlan to wan forwarding policy" -j zone_wan_dest_ACCEPT
[2174:130360] -A zone_pvtlan_forward -m comment --comment "!fw3: Zone pvtlan to IOT forwarding policy" -j zone_IOT_dest_ACCEPT
[0:0] -A zone_pvtlan_forward -m conntrack --ctstate DNAT -m comment --comment "!fw3: Accept port forwards" -j ACCEPT
[5:260] -A zone_pvtlan_forward -m comment --comment "!fw3" -j zone_pvtlan_dest_ACCEPT
[0:0] -A zone_pvtlan_forward -m conntrack --ctstate DNAT -m comment --comment "!fw3: Accept port forwards" -j ACCEPT
[5:260] -A zone_pvtlan_forward -m comment --comment "!fw3" -j zone_pvtlan_dest_ACCEPT
[51328:3325606] -A zone_pvtlan_input -m comment --comment "!fw3: Custom pvtlan input rule chain" -j input_pvtlan_rule
[51328:3325606] -A zone_pvtlan_input -m comment --comment "!fw3: Custom pvtlan input rule chain" -j input_pvtlan_rule
[0:0] -A zone_pvtlan_input -p gre -m comment --comment "!fw3: Allow-GRE" -j ACCEPT
[0:0] -A zone_pvtlan_input -m conntrack --ctstate DNAT -m comment --comment "!fw3: Accept port redirections" -j ACCEPT
[51328:3325606] -A zone_pvtlan_input -m comment --comment "!fw3" -j zone_pvtlan_src_ACCEPT
[0:0] -A zone_pvtlan_input -m conntrack --ctstate DNAT -m comment --comment "!fw3: Accept port redirections" -j ACCEPT
[75:3900] -A zone_pvtlan_input -m comment --comment "!fw3" -j zone_pvtlan_src_ACCEPT
[299:45226] -A zone_pvtlan_output -m comment --comment "!fw3: Custom pvtlan output rule chain" -j output_pvtlan_rule
[299:45226] -A zone_pvtlan_output -m comment --comment "!fw3: Custom pvtlan output rule chain" -j output_pvtlan_rule
[299:45226] -A zone_pvtlan_output -m comment --comment "!fw3" -j zone_pvtlan_dest_ACCEPT
[0:0] -A zone_pvtlan_output -m comment --comment "!fw3" -j zone_pvtlan_dest_ACCEPT
[51253:3321706] -A zone_pvtlan_src_ACCEPT -i br-pvtlan -m conntrack --ctstate NEW,UNTRACKED -m comment --comment "!fw3" -j ACCEPT
[0:0] -A zone_vpnMF_dest_ACCEPT -o tun0 -m conntrack --ctstate INVALID -m comment --comment "!fw3: Prevent NAT leakage" -j DROP
[0:0] -A zone_vpnMF_dest_ACCEPT -o tun0 -m comment --comment "!fw3" -j ACCEPT
[0:0] -A zone_vpnMF_forward -m comment --comment "!fw3: Custom vpnMF forwarding rule chain" -j forwarding_vpnMF_rule
[0:0] -A zone_vpnMF_forward -m conntrack --ctstate DNAT -m comment --comment "!fw3: Accept port forwards" -j ACCEPT
[0:0] -A zone_vpnMF_forward -m comment --comment "!fw3" -j zone_vpnMF_dest_ACCEPT
[0:0] -A zone_vpnMF_input -m comment --comment "!fw3: Custom vpnMF input rule chain" -j input_vpnMF_rule
[0:0] -A zone_vpnMF_input -m conntrack --ctstate DNAT -m comment --comment "!fw3: Accept port redirections" -j ACCEPT
[0:0] -A zone_vpnMF_input -m comment --comment "!fw3" -j zone_vpnMF_src_DROP
[0:0] -A zone_vpnMF_output -m comment --comment "!fw3: Custom vpnMF output rule chain" -j output_vpnMF_rule
[0:0] -A zone_vpnMF_output -m comment --comment "!fw3" -j zone_vpnMF_dest_ACCEPT
[0:0] -A zone_vpnMF_src_DROP -i tun0 -m comment --comment "!fw3" -j DROP
[807:42237] -A zone_wan_dest_ACCEPT -o pppoe-wan -m conntrack --ctstate INVALID -m comment --comment "!fw3: Prevent NAT leakage" -j DROP
[61343:6883041] -A zone_wan_dest_ACCEPT -o pppoe-wan -m comment --comment "!fw3" -j ACCEPT
[501:26270] -A zone_wan_dest_ACCEPT -o eth0.5 -m conntrack --ctstate INVALID -m comment --comment "!fw3: Prevent NAT leakage" -j DROP
[23613:3636876] -A zone_wan_dest_ACCEPT -o eth0.5 -m comment --comment "!fw3" -j ACCEPT
[0:0] -A zone_wan_dest_REJECT -o pppoe-wan -m comment --comment "!fw3" -j reject
[0:0] -A zone_wan_dest_REJECT -o eth0.5 -m comment --comment "!fw3" -j reject
[0:0] -A zone_wan_forward -m comment --comment "!fw3: Custom wan forwarding rule chain" -j forwarding_wan_rule
[0:0] -A zone_wan_forward -m conntrack --ctstate DNAT -m comment --comment "!fw3: Accept port forwards" -j ACCEPT
[0:0] -A zone_wan_forward -m comment --comment "!fw3" -j zone_wan_dest_REJECT
[984:86687] -A zone_wan_input -m comment --comment "!fw3: Custom wan input rule chain" -j input_wan_rule
[0:0] -A zone_wan_input -p udp -m udp --dport 68 -m comment --comment "!fw3: Allow-DHCP-Renew" -j ACCEPT
[0:0] -A zone_wan_input -p icmp -m icmp --icmp-type 8 -m comment --comment "!fw3: Allow-Ping" -j ACCEPT
[0:0] -A zone_wan_input -p igmp -m comment --comment "!fw3: Allow-IGMP" -j ACCEPT
[0:0] -A zone_wan_input -p udp -m udp --dport 51820 -m comment --comment "!fw3: wireguard" -j ACCEPT
[0:0] -A zone_wan_input -m conntrack --ctstate DNAT -m comment --comment "!fw3: Accept port redirections" -j ACCEPT
[984:86687] -A zone_wan_input -m comment --comment "!fw3" -j zone_wan_src_REJECT
[17100:1347015] -A zone_wan_output -m comment --comment "!fw3: Custom wan output rule chain" -j output_wan_rule
[17100:1347015] -A zone_wan_output -m comment --comment "!fw3" -j zone_wan_dest_ACCEPT
[929:73876] -A zone_wan_src_REJECT -i pppoe-wan -m comment --comment "!fw3" -j reject
[55:12811] -A zone_wan_src_REJECT -i eth0.5 -m comment --comment "!fw3" -j reject
[2:212] -A zone_wgMF_dest_ACCEPT -o wg0 -m comment --comment "!fw3" -j ACCEPT
[2:212] -A zone_wgMF_dest_ACCEPT -o wg1 -m comment --comment "!fw3" -j ACCEPT
[1:84] -A zone_wgMF_dest_ACCEPT -o bond0 -m comment --comment "!fw3" -j ACCEPT
[0:0] -A zone_wgMF_dest_REJECT -o wg0 -m comment --comment "!fw3" -j reject
[0:0] -A zone_wgMF_dest_REJECT -o wg1 -m comment --comment "!fw3" -j reject
[0:0] -A zone_wgMF_dest_REJECT -o bond0 -m comment --comment "!fw3" -j reject
[128:7680] -A zone_wgMF_forward -m comment --comment "!fw3: Custom wgMF forwarding rule chain" -j forwarding_wgMF_rule
[128:7680] -A zone_wgMF_forward -m comment --comment "!fw3: Zone wgMF to pvtlan forwarding policy" -j zone_pvtlan_dest_ACCEPT
[128:7680] -A zone_wgMF_forward -m comment --comment "!fw3: Zone wgMF to IOT forwarding policy" -j zone_IOT_dest_ACCEPT
[0:0] -A zone_wgMF_forward -m conntrack --ctstate DNAT -m comment --comment "!fw3: Accept port forwards" -j ACCEPT
[0:0] -A zone_wgMF_forward -m comment --comment "!fw3" -j zone_wgMF_dest_REJECT
[9:954] -A zone_wgMF_input -m comment --comment "!fw3: Custom wgMF input rule chain" -j input_wgMF_rule
[0:0] -A zone_wgMF_input -m conntrack --ctstate DNAT -m comment --comment "!fw3: Accept port redirections" -j ACCEPT
[9:954] -A zone_wgMF_input -m comment --comment "!fw3" -j zone_wgMF_src_ACCEPT
[5:508] -A zone_wgMF_output -m comment --comment "!fw3: Custom wgMF output rule chain" -j output_wgMF_rule
[5:508] -A zone_wgMF_output -m comment --comment "!fw3" -j zone_wgMF_dest_ACCEPT
[5:530] -A zone_wgMF_src_ACCEPT -i wg0 -m conntrack --ctstate NEW,UNTRACKED -m comment --comment "!fw3" -j ACCEPT
[4:424] -A zone_wgMF_src_ACCEPT -i wg1 -m conntrack --ctstate NEW,UNTRACKED -m comment --comment "!fw3" -j ACCEPT
[0:0] -A zone_wgMF_src_ACCEPT -i bond0 -m conntrack --ctstate NEW,UNTRACKED -m comment --comment "!fw3" -j ACCEPT
COMMIT
For this specific use case, it may just be easier to setup a routing table with appropriate ip rules and ip routes. In your other thread, I linked you my github repository where I posted the setup for a dual bonded internet link.
This setup had two wan links, with 2 openvpn instances, each one routing over one and only one of the wan links. Which is precisely what you're trying to do.
80%+ of that code is likely to be re-usable for your use case. It's even got logic to read the configuration from a config file.
I strongly suggest you look at it for inspiration. In the "bonding" init script, you will find a set of functions that setup the routing correctly for two physical links (add_source_route).
Pretty much everything you need is there, from init scripts to hotplug scripts to config files that show how to do what you're trying to do. They'll need a little bit of modification, mostly to remove the openvpn logic. Your wireguard interfaces are setup by Openwrt, so you won't need to add much logic, just remove the openvpn stuff
I originally added the iperf3 caveat stuff, however I think in the context of iperf3 it now actually supports SO_BINDTODEVICE which it didn't before and this was the main reason why it doesn't play nice with mwan3. It did go into the iperf3 codebase https://github.com/esnet/iperf/pull/1097, but I don't know if it's made it into an actual release yet. There doesn't seem to have been an official release since mid 2020 and the SO_BINDTODEVICE changes went in late 2020.
Newer versions of mwan3 should resolve the issue of binding with the mwan3 use command, although setting specific rules either to fallback to the main routing table or force a specific interface should work.