Mwan3 port to nftables

Did anyone get the chance to look at the preview? Any opinions?

This is your chance to have a say otherwise I'll choose for everyone. See original post above

Should we expect it to say the traffic will use privado?

It should yes. This is a bug I've just fixed - I see you have a timer enabled and the set element matching logic in the traffic simulator doesn't consider that. Same problem for counter enabled sets.

I've fixed it already and will deploy in next release.

1 Like


This needs some cosmetic improvements, like those “arrows” glitch, but overall it is much better than original version

That's intentional, not a glitch. Meant to show failover configuration. Curious why you considered it a glitch, or is it just that you'd favour a different visual representation, and if so, what?

And what about the notion of completely removing the Members tab and hiding the implementation details (members / weights /metrics) in the luci app, since the policy builder handles it all automagically?

New version requirements has shown that I didn't have ip-full. Will be testing it out later on if I manage.

root@wrt:/tmp# apk add mwan3-3.5.2-1_openwrt-25.12_aarch64_generic.apk --allow-untrusted
(1/5) Purging ip-tiny (6.18.0-r2)
  Executing ip-tiny-6.18.0-r2.pre-deinstall
  * add alternative: /sbin/ip -> /bin/busybox
(2/5) Installing libelf1 (0.192-r1)
  Executing libelf1-0.192-r1.post-install
(3/5) Installing libbpf1 (1.6.2-r1)
  Executing libbpf1-1.6.2-r1.post-install
(4/5) Installing ip-full (6.18.0-r2)
  Executing ip-full-6.18.0-r2.post-install
  * add alternative: /sbin/ip -> /usr/libexec/ip-full
(5/5) Upgrading mwan3 (3.5.1-r1 -> 3.5.2-r1)
  Executing mwan3-3.5.2-r1.pre-upgrade
  Installing file to etc/mwan3.user.apk-new
  Installing file to etc/config/mwan3.apk-new
  Executing mwan3-3.5.2-r1.post-upgrade
  * sh: out of range
  * sh: out of range
  * sh: out of range
  * sh: out of range
  * sh: out of range
  * sh: out of range
  * sh: out of range
  * sh: out of range
  * sh: out of range
  * sh: out of range
  * sh: out of range
  * sh: out of range
  * sh: out of range
  * sh: out of range
  * sh: out of range
  * sh: out of range
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * sh: out of range
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * sh: out of range
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * sh: out of range
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * sh: out of range
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
  * RTNETLINK answers: File exists
OK: 97.8 MiB in 313 packages
root@wrt:/tmp# 

Most importantly, please don’t take this too seriously — it’s just my non-professional view of things, :grinning_face_with_smiling_eyes:
These arrows looked a bit misleading to me, as if traffic from one WAN is supposed to go into the other. But, maybe it would be enough to name the columns IPv4 priority order and IPv6 priority order, this changes a lot, for the user first time visiting this tab.

It’s hard for me to say much here. Of course, there will always be someone who absolutely needs the old interface.

On the other hand... maybe the Members tab could be removed, and the information about policy members could be shown somewhere in MultiWAN Manager → Troubleshooting, in case someone needs it for analysis.

That's actually quite a good suggestion. Frames the immediate context differently so someone seeing it realises this is in fact a priority order.

I think the load balancing format is self-explanatory. I chose the formats so we could represent something like (wan 70%, wan2 30%) --> wan3, ie., a 70/30 balanced policy on wan and wan2 that fails over to wan3.

Happy to consider a different visual representation if there's a more intuitive way of representing it that anyone can propose? Or even a different symbol to the arrow?

Something like the following might also work.

(wan 70%, wan2 30%) : wan3 or [wan 70%, wan2 30%] : [wan3]

I don't actually have to implement anything to retain it. I could perhaps hide it by default and only show it if a user checks an "Advanced options" checkbox? There's also the option for advanced users to just use uci commands or edit the config directly. I think perhaps hiding it and making available via a checkbox option might be a good compromise. Removes nothing permanently, hides the complexity from new users.

Yeah, there's quite a substantial difference between ip-tiny and ip-full and I only clicked belatedly that this might in fact causes issues. Better to force the dependency on ip-full.

I see now that you're running ip-full you're also getting the RTNETLINK errors, but I'm still having trouble finding where they're coming from, since the calls in mwan3 are guarded with redirections.

Like the sh: out of range errors, do these appear for you if you do a manual stop and start of mwan3 from the command line? I simply don't see any of these on either my x86_64 glibc custom build or other qemu test platforms.

Perhaps the thing to do is to stop mwan3, then add a set -x to the top of the init.d script, then run it and pipe stderr and stdout to a file so I can see where it's coming from...

It also belatedly occurred to me that this might be caused by stuff in your /etc/mwan3.user. If you have code there, could you temporarily put an exit 0 at the top of the file to prevent execution of any of that code and test like that too.

This one looks good too.

Great idea, I think. :+1:

So current candidates are

1.

(wan 70%, wan2 30%) --> wan3 --> wan4     # balance 70/30 = 2 failovers
wan --> wan2                              # failover wan to wan2

2.

(wan 70%, wan2 30%) --> (wan3) --> (wan4) # balance 70/30 = 2 failovers
(wan) --> (wan2)                          # failover wan to wan2

3.

[wan 70%, wan2 30%] : [wan3] : [wan4]     # balance 70/30 = 2 failovers
[wan] :  [wan2]                           # failover wan to wan2

4.

[wan 70%, wan2 30%] --> [wan3] --> [wan4] # balance 70/30 = 2 failovers
[wan] -->  [wan2]                         # failover wan to wan2

Any other candidates, anyone?

exit 0 at beginning of mwan3.user didn't change anything.
here's the -x output +/- 10 lines, hope it helps.

# grep -C 10 "out of range" /tmp/mwan3.log
+ _json_inc JSON_SEQ seq
+ let 'JSON_SEQ += 1' 'seq = JSON_SEQ'
+ local 'table=J_A13'
+ _json_set_var U_J_A13 J_T12
+ local '___val=J_T12'
+ eval 'U_J_A13="$___val"'
+ U_J_A13=J_T12
+ export -- 'K_J_A13='
+ unset S_J_A13
+ _json_set_var JSON_CUR J_A13
sh: out of range
+ local '___val=J_A13'
+ eval 'JSON_CUR="$___val"'
+ JSON_CUR=J_A13
+ _jshn_append JSON_UNSET J_A13
+ local '_a_value=J_A13'
+ eval 'JSON_UNSET="${JSON_UNSET}${JSON_UNSET:+ }$_a_value"'
+ JSON_UNSET='J_V_up J_V_pending J_V_available J_V_autostart J_V_dynamic J_V_uptime J_V_l3_device J_V_proto J_A1 J_V_updated J_A1_1 J_A1_2 J_V_metric J_V_dns_metric J_V_delegation J_A2 J_V_ipv4_address J_T3 J_A2_1 J_T3_address J_T3_mask J_A4 J_V_ipv6_address J_A5 J_V_ipv6_prefix J_A6 J_V_ipv6_prefix_assignment J_A7 J_V_route J_T8 J_A7_1 J_T8_target J_T8_mask J_T8_nexthop J_T8_source J_A9 J_V_dns_server J_A10 J_V_dns_search J_A11 J_V_neighbors J_T12 J_V_inactive J_A13'
+ _json_add_generic array ipv4_address J_A13 J_T12
+ local var
+ '[' J_T '=' J_A ]
--
+ local 'cmd=running'
+ local 'help=Check if service is running'
+ ubus -S call network.interface.henet status
+ printf '%-16s%s' running 'Check if service is running'
+ local 'extra=running         Check if service is running'
+ ALL_HELP='\tstart           Start the service\n\tstop            Stop the service\n\trestart         Restart the service\n\treload          Reload configuration files (or restart if service does not implement reload)\n\tenable          Enable service autostart\n\tdisable         Disable service autostart\n\tenabled         Check if service is started on boot\n\trunning         Check if service is running\n'
+ ALL_COMMANDS='boot shutdown depends start stop restart reload enable disable enabled running'
+ extra_command status 'Service status'
+ local 'cmd=status'
+ local 'help=Service status'
sh: out of range
+ printf '%-16s%s' status 'Service status'
+ local 'extra=status          Service status'
+ ALL_HELP='\tstart           Start the service\n\tstop            Stop the service\n\trestart         Restart the service\n\treload          Reload configuration files (or restart if service does not implement reload)\n\tenable          Enable service autostart\n\tdisable         Disable service autostart\n\tenabled         Check if service is started on boot\n\trunning         Check if service is running\n\tstatus          Service status\n'
+ ALL_COMMANDS='boot shutdown depends start stop restart reload enable disable enabled running status'
+ extra_command trace 'Start with syscall trace'
+ local 'cmd=trace'
+ local 'help=Start with syscall trace'
+ printf '%-16s%s' trace 'Start with syscall trace'
+ local 'extra=trace           Start with syscall trace'
+ ALL_HELP='\tstart           Start the service\n\tstop            Stop the service\n\trestart         Restart the service\n\treload          Reload configuration files (or restart if service does not implement reload)\n\tenable          Enable service autostart\n\tdisable         Disable service autostart\n\tenabled         Check if service is started on boot\n\trunning         Check if service is running\n\tstatus          Service status\n\ttrace           Start with syscall trace\n'
--
+ K_J_T6='address mask class'
+ json_add_object assigned
+ _json_add_table assigned object T
+ local cur seq
+ _json_get_var cur JSON_CUR
+ eval 'cur="$JSON_CUR"'
+ cur=J_T6
+ _json_inc JSON_SEQ seq
+ let 'JSON_SEQ += 1' 'seq = JSON_SEQ'
+ local 'table=J_T7'
sh: out of range
+ PROCD_RELOAD_DELAY=1000
+ _PROCD_SERVICE=
+ _json_set_var U_J_T7 J_T6
+ local '___val=J_T6'
+ eval 'U_J_T7="$___val"'
+ U_J_T7=J_T6
+ export -- 'K_J_T7='
+ unset S_J_T7
+ _json_set_var JSON_CUR J_T7
+ local '___val=J_T7'
--
+ mwan3_get_true_iface true_iface netassist
+ local family V
+ _true_iface=netassist
+ config_get family netassist family ipv4
+ eval export -n -- 'family=${CONFIG_netassist_family:-${4}}'
+ export -n -- 'family=ipv6'
+ '[' ipv6 '=' ipv4 ]
+ '[' ipv6 '=' ipv6 ]
+ V=6
+ ubus call network.interface.netassist_6 status
sh: out of range
+ export 'true_iface=netassist'
+ ubus -S call network.interface.netassist status
+ status=
+ '[' -n  ]
+ '['  '!=' 1 ]
+ return
+ config_get cfgtype wan_lte_only TYPE
+ eval export -n -- 'cfgtype=${CONFIG_wan_lte_only_TYPE:-${4}}'
+ export -n -- 'cfgtype=policy'
+ '[' -n interface ]
--
+ '[' -n interface ]
+ '[' xmember '!=' xinterface ]
+ continue
+ config_get cfgtype wg_vpn_only TYPE
+ eval export -n -- 'cfgtype=${CONFIG_wg_vpn_only_TYPE:-${4}}'
+ export -n -- 'cfgtype=policy'
+ '[' -n interface ]
+ '[' xpolicy '!=' xinterface ]
+ continue
+ wait 9076 9087 9121 9183 9263 9340 9487
sh: out of range
+ . /lib/functions/network.sh
+ . /lib/mwan3/mwan3.sh
+ . /usr/share/libubox/jshn.sh
+ . /lib/mwan3/common.sh
+ IP4='ip -4'
+ IP6='ip -6'
+ basename /etc/rc.common
+ SCRIPTNAME=rc.common
+ MWAN3_STATUS_DIR=/var/run/mwan3
+ MWAN3TRACK_STATUS_DIR=/var/run/mwan3track
--
+ ALL_HELP='\tstart           Start the service\n\tstop            Stop the service\n\trestart         Restart the service\n\treload          Reload configuration files (or restart if service does not implement reload)\n\tenable          Enable service autostart\n\tdisable         Disable service autostart\n\tenabled         Check if service is started on boot\n\trunning         Check if service is running\n\tstatus          Service status\n\ttrace           Start with syscall trace\n\tinfo            Dump procd service info\n'
+ ALL_COMMANDS='boot shutdown depends start stop restart reload enable disable enabled running status trace info'
+ . /lib/functions/procd.sh
+ . /usr/share/libubox/jshn.sh
+ PROCD_RELOAD_DELAY=1000
+ _PROCD_SERVICE=
+ _procd_wrapper procd_open_service procd_close_service procd_add_instance procd_add_raw_trigger procd_add_config_trigger procd_add_interface_trigger procd_add_mount_trigger procd_add_reload_trigger procd_add_reload_data_trigger procd_add_reload_interface_trigger procd_add_action_mount_trigger procd_add_reload_mount_trigger procd_add_restart_mount_trigger procd_open_trigger procd_close_trigger procd_open_instance procd_close_instance procd_open_validate procd_close_validate procd_add_jail procd_add_jail_mount procd_add_jail_mount_rw procd_set_param procd_append_param procd_add_validation procd_set_config_changed procd_kill procd_send_signal
+ procd_lock
+ readlink /etc/init.d/mwan3
+ local 'basescript='
sh: out of range
+ basename /etc/init.d/mwan3
+ local 'service_name=mwan3'
+ flock -n 1000
+ '[' 0 '!=' 0 ]
+ '[' -n procd_open_service ]
+ eval 'procd_open_service() { _procd_call _procd_open_service "$@"; }'
+ shift
+ '[' -n procd_close_service ]
+ eval 'procd_close_service() { _procd_call _procd_close_service "$@"; }'
+ shift
--
+ list_contains ALL_COMMANDS running
+ local 'var=ALL_COMMANDS'
+ local 'str=running'
+ local val
+ eval 'val=" ${ALL_COMMANDS} "'
+ val=' boot shutdown depends start stop restart reload enable disable enabled running status trace info  '
+ '[' ' boot shutdown depends start stop restart reload enable disable enabled' '!=' ' boot shutdown depends start stop restart reload enable disable enabled running status trace info  ' ]
+ running
+ service_running
+ '[' -d /var/run/mwan3 ]
sh: out of range
sh: out of range
sh: out of range
sh: out of range
+ mwan3_set_general_nft
+ local chain_exists restore_vmap save_vmap all_marks
+ '[' 0 -eq 0 ]
+ nft list chain inet mwan3 mwan3_prerouting
+ grep -c 'meta mark'
+ chain_exists=0
+ '[' 0 -gt 0 ]
+ mwan3_build_or_chains_nft
+ local id mark suffix want
+ want=
root@wrt:/tmp# 

I vote for this one

It certainly narrows it down: the errors are asynchronous from concurrent background processes as they appear somewhat randomly between unrelated trace lines (jshn string ops, extra_command registrations, variable assignments) with nothing happening in the visible output that could produce it.

So it's probably from the 7 background hotplug_startup & processes writing to stderr while the main script's trace is printing.

Next step: remove the set -x from the init.d script and put it at the top of 25-mwan3 and capture the log again.

Turns out hiding the tab is actually rather hard to do: it's all server rendered and baked before the javascript on the app gets any say, so it's like playing a game of whack-a-mole to get it to hide and stay hidden. Best to leave it visible and put a note that says the policy tab handles it all and only make edits here if you need custom metrics and/or weights

Here it is, -x was added in /etc/hotplug.d/iface/25-mwan3

# grep -C 10 "out of range" /tmp/3mwan.log
+ '[' xrule+  '!='export xinterface 'device=eth1' ]

+ continue
+ config_get cfgtype+  wan_lte'[' TYPE -z
 eth1 ]
+ + evalconfig_get_bool export enabled -n wan --+  enabled 'cfgtype=${CONFIG_wan_lte_TYPE:-${4}}'
'['
  -eq 0 ]+
local+  _tmpexport
 -n -- 'cfgtype=interface'
sh: out of range
+ + config_get'[' _tmp -n wan interface enabled ]

+ '[' xinterface '!=' xinterface ]+
eval export -n --+  '_tmp=${CONFIG_wan_enabled:-${4}}'eval
 'update_table "$section" "$@"'
+ + exportupdate_table -n wan_lte --
 '_tmp=1'
+ local family curr_table device enabled
+ let _tid++
+ config_get family wan_lte family ipv4
--
+ config_get _tmp wg_vps enabled
+ eval export -n -- '_tmp=${CONFIG_wg_vps_enabled:-${4}}'
+ export -n -- '_tmp='
+ get_bool
+ local '_tmp='
+ _tmp=
+ echo -n
+ _tmp=
+ export -n 'enabled='
+ '['  -eq 0 ]
sh: out of range
+ eval 'echo     "$mwan3_dev_tbl_ipv4"'
+ echo ' eth1=1 VLANs.1=2 vpn_unl=3 '
+ curr_table=' eth1=1 VLANs.1=2 vpn_unl=3 '
+ export 'mwan3_dev_tbl_ipv4= eth1=1 VLANs.1=2 vpn_unl=3 wg_vps=4 '
+ config_get cfgtype wg_vpn TYPE
+ eval export -n -- 'cfgtype=${CONFIG_wg_vpn_TYPE:-${4}}'
+ export -n -- 'cfgtype=interface'
+ '[' -n interface ]
+ '[' xinterface '!=' xinterface ]
+ eval 'update_table "$section" "$@"'
--
+ _tmp=1
+ export -n 'enabled=1'
+ '[' 1 -eq 0 ]
+ mwan3_id2mask id MMX_MASK
+ local bit_msk bit_val result
+ bit_val=0
+ result=0
+ _tmp=
+ export -n 'enabled='
+ '['  -eq 0 ]
sh: out of range
+ eval 'echo     "$mwan3_dev_tbl_ipv4"'
+ echo ' eth1=1 VLANs.1=2 vpn_unl=3 wg_vps=4 '
+ eval 'echo     "$mwan3_dev_tbl_ipv6"'
+ echo ' '
+ curr_table=' '
+ __tmp='export device='"'"'wg_vps'"'"'; '+ +
export 'mwan3_dev_tbl_ipv6= 6in4-henet=7 '
+ '[' -z 'export device='"'"'wg_vps'"'"'; ' ]
+ config_get cfgtype netassist+  TYPE
eval 'export device='"'"'wg_vps'"'"'; '
--
+ eval export -n -- 'cfgtype=${CONFIG_wg_awh_TYPE:-${4}}'
+ export -n -- 'cfgtype=interface'
+ '[' -n interface ]
+ '[' xinterface '!=' xinterface+  ]_tmp=

+ eval+  'update_table "$section" "$@"'
export -n 'enabled='
+ update_table wg_awh
+ '[' +  -eqlocal 0 family ] curr_table
 device enabled
sh: out of range
+ jsonfilter -s+ let _tid++
+ config_get family wg_awh family ipv4
+ eval export -n -- 'family=${CONFIG_wg_awh_family:-${4}}'
+ export -n -- 'family=ipv4'
+ network_get_device device wg_awh
+ __network_ifstatus device wg_awh .l3_device
 '{
        "interface": [
                {
                        "interface": "adsb",
--
                }
        ]
}}' -e 'device=@.interface[@.interface='"'"'wan_lte'"'"'].l3_device'
+ error=
+ rm -f /tmp/mwan3_nft_batch.6029
+ + _tmp=
eval 'echo       "$mwan3_dev_tbl_ipv4"'
+ export -n 'enabled='
+ + echo'[' ' eth1=1 VLANs.1=2 vpn_unl=3 wg_vps=4 wg_vpn=5 '
 -eq 0 ]
sh: out of range
+ + curr_table=' eth1=1 VLANs.1=2 vpn_unl=3 wg_vps=4 wg_vpn=5 '
eval 'echo       "$mwan3_dev_tbl_ipv6"'
+ export 'mwan3_dev_tbl_ipv4= eth1=1 VLANs.1=2 vpn_unl=3 wg_vps=4 wg_vpn=5 wg_awh=6 '
+ echo ' '
+ config_get cfgtype henet TYPE
+ eval export -n -- 'cfgtype=${CONFIG_henet_TYPE:-${4}}'
+ export -n -- 'cfgtype=interface'
+ '[' -n interface ]
+ '[' xinterface '!=' xinterface ]
+ eval 'update_table "$section" "$@"'
--
                        "data": {

                        }
                }
        ]
}}' -e 'device=@.interface[@.interface='"'"'vpn_unl'"'"'].l3_device'
+ _tmp=
+ export -n 'enabled='
+ '['  -eq 0+  ]
curr_table=' eth1=1 VLANs.1=2 vpn_unl=3 wg_vps=4 wg_vpn=5 wg_awh=6 '
sh: out of range
+ '[' eth1 '=' VLANs.1 ]
+ '[' VLANs.1 '=' VLANs.1 ]
+ _tid=2
+ export 'tid=2'
+ return
+ '[' -z '10.0.y.y/24 dev VLANs.1 proto static scope link metric 20' ]
+ '[' -z '10.0.y.y/24 dev VLANs.1 proto static scope link metric 20' ]
+ '[' -z 2 ]
+ '[' 2 '=' 1 ]
+ read -r route_line
--
+ '[' 6 '=' 1 ]
+ read -r route_line
+ mwan3_route_line_dev tid '10.0.x.x dev wg_vps proto static scope link metric 901' ipv4
+ local _tid route_line route_device route_family entry curr_table
+ route_line='10.0.x.x dev wg_vps proto static scope link metric 901'
+ route_family=ipv4
+ echo '10.0.0.0/8 dev wg_vpn proto static scope link metric 202'
+ _tmp=
+ export -n 'enabled='
+ '['  -eq 0 ]
sh: out of range
+ sed -ne 's/.*dev \([^ ]*\).*/\1/p'
+ + evalmwan3_route_line_dev 'echo       "$mwan3_dev_tbl_ipv4"' tid
 '10.0.0.0/8 dev wg_vpn proto static scope link metric 202' ipv4
+ local _tid route_line route_device route_family entry+  curr_table
echo ' eth1=1 VLANs.1=2 vpn_unl=3 '
+ route_line='10.0.0.0/8 dev wg_vpn proto static scope link metric 202'
+ route_family=ipv4
+ curr_table=' eth1=1 VLANs.1=2 vpn_unl=3 '
+ + echoexport '10.0.x.x dev wg_vps proto static scope link metric 901' 'mwan3_dev_tbl_ipv4= eth1=1 VLANs.1=2 vpn_unl=3 wg_vps=4 '

--
+ echo '10.x.y.z dev vpn_unl proto static scope link metric 201'
+ get_bool
+ local '_tmp='
+ _tmp=
+ echo -n
+ _tmp=
+ eval 'echo "$mwan3_dev_tbl_ipv4"'
+ export -n 'enabled='
+ '['  -eq 0 ]+
echo ' eth1=1 VLANs.1=2 vpn_unl=3 wg_vps=4 wg_vpn=5 wg_awh=6 '
sh: out of range
+ sed -ne 's/.*dev \([^ ]*\).*/\1/p'
+ route_device=wg_awh
+ + unset tidcurr_table=' eth1=1 VLANs.1=2 vpn_unl=3 wg_vps=4 wg_vpn=5 wg_awh=6 '

+ '[' -z wg_awh ]
+ '[' eth1 '=' wg_awh ]
+ '[' VLANs.1 '=' wg_awh ]
+ '[' vpn_unl '=' wg_awh ]
+ '[' wg_vps '=' wg_awh ]
+ '[' wg_vpn '=' wg_awh ]
--
+ _tmp=
+ echo -n
+ echo '10.0.x.x dev wg_vps proto static scope link metric 901'
+ route_device=VLANs.30
+ unset tid
+ '[' -z VLANs.30 ]
+ _tmp=
+ sed -ne 's/.*dev \([^ ]*\).*/\1/p'
+ export -n 'enabled='
+ '['  -eq 0 ]
sh: out of range
+ eval 'echo "$mwan3_dev_tbl_ipv4"'
+ echo ' eth1=1 VLANs.1=2 vpn_unl=3 wg_vps=4 wg_vpn=5 wg_awh=6 '
+ curr_table=' eth1=1 VLANs.1=2 vpn_unl=3 wg_vps=4 wg_vpn=5 wg_awh=6 '
+ '[' eth1 '=' VLANs.30 ]
+ '[' VLANs.1 '=' VLANs.30 ]
+ '[' vpn_unl '=' VLANs.30 ]
+ '[' wg_vps '=' VLANs.30 ]
+ '[' wg_vpn '=' VLANs.30 ]
+ '[' wg_awh '=' VLANs.30 ]
+ '[' -z '192.168.30.0/24 dev VLANs.30 proto kernel scope link src 192.168.30.1' ]
--
+ echo -n
+ eval 'echo "$mwan3_dev_tbl_ipv4"'
+ echo ' eth1=1 VLANs.1=2 vpn_unl=3 wg_vps=4 wg_vpn=5 wg_awh=6 '
+ + _tmp=route_device=VLANs.7

+ + unsetexport tid -n
 'enabled='
+ '[' -z VLANs.7 ]
+ '['  -eq+  0 ]curr_table=' eth1=1 VLANs.1=2 vpn_unl=3 wg_vps=4 wg_vpn=5 wg_awh=6 '

sh: out of range
+ '[' eth1 '=' VLANs.20 ]
+ '[' VLANs.1 '=' VLANs.20 ]
+ '[' vpn_unl '=' VLANs.20 ]
+ '[' wg_vps '=' VLANs.20 ]
+ '[' wg_vpn '=' VLANs.20 ]
+ '[' wg_awh '=' VLANs.20 ]
+ '[' -z '192.168.20.0/24 dev VLANs.20 proto kernel scope link src 192.168.20.1' ]
+ '[' -z '192.168.20.0/24 dev VLANs.20 proto kernel scope link src 192.168.20.1' ]
+ '[' -z  ]
+ '[' -n '

Ok, that helped me to find it, thanks.

It's due to not having an enabled option for your interfaces in your mwan3 config and the code not putting in a default initialiser. The error would not have affected your functionality.

The errored test exits non-zero (2) anyway and so the && return would not execute.

Put a 1 at the end of line 96 in/lib/mwan3/mwan3.sh to give it the initial default if it's not in the config and the unsightly error should go away. I'll include the fix in the next release

--- a/net/mwan3/files/lib/mwan3/mwan3.sh
+++ b/net/mwan3/files/lib/mwan3/mwan3.sh
@@ -93,7 +93,7 @@ mwan3_update_dev_to_table()
                config_get family "$1" family ipv4
                network_get_device device "$1"
                [ -z "$device" ] && return
-               config_get_bool enabled "$1" enabled
+               config_get_bool enabled "$1" enabled 1
                [ "$enabled" -eq 0 ] && return
                curr_table=$(eval "echo  \"\$mwan3_dev_tbl_${family}\"")
                export "mwan3_dev_tbl_$family=${curr_table}${device}=$_tid "

While you're at it, also edit line 837. Change the ip route add operation to a replace operation and remove the || LOG debug....

I expect you are likely to see the RTNETLINK errors vanish too.

@@ -834,8 +834,7 @@ mwan3_create_iface_route()
                        # possible that routes are already in the table
                        # if 'connected' was called after 'ifup'
                        [ -n "$tbl" ] && [ -z "${tbl##*$route_line$'\n'*}" ] && continue
-                       $IP route add table $id $route_line ||
-                               LOG debug "Route '$route_line' already added to table $id"
+                       $IP route replace table $id $route_line
                fi
 
        done
1 Like

And you're right, error disappeared:

root@wrt:~# /etc/init.d/mwan3 stop
root@wrt:~# /etc/init.d/mwan3 start
root@wrt:~# 
2 Likes

@woffko curious to know if this change fixes your own RTNETLINK errors