Mwan3 udp data cannot be sourced in and out

[wtwind]

BUG mwan3负载均衡双wan口外网访问wan1口udp数据没有原路返回走向了wan2口，tcp数据正常.BUG mwan3 load balancing dual wan ports external network access wan1 port udp data does not return to the original path to wan2 port, tcp data is normal.

root@OpenWrt:~# ip rule
0:      from all lookup local
1000:   from all fwmark 0x1 lookup 100
1001:   from all iif pppoe-wan lookup 1
1002:   from all iif pppoe-vwan0 lookup 2
2001:   from all fwmark 0x100/0x3f00 lookup 1
2002:   from all fwmark 0x200/0x3f00 lookup 2
2061:   from all fwmark 0x3d00/0x3f00 blackhole
2062:   from all fwmark 0x3e00/0x3f00 unreachable
3001:   from all fwmark 0x100/0x3f00 unreachable
3002:   from all fwmark 0x200/0x3f00 unreachable
32766:  from all lookup main
32767:  from all lookup default
root@OpenWrt:~# ip route show table 1
default via 175.0.80.1 dev pppoe-wan proto static metric 20
10.10.1.0/24 dev eth3 proto kernel scope link src 10.10.1.100
10.10.10.0/24 dev br-lan proto kernel scope link src 10.10.10.1
10.10.11.0/24 dev wg0 proto kernel scope link src 10.10.11.1
10.10.11.2 dev wg0 proto static scope link
10.10.11.3 dev wg0 proto static scope link
10.10.11.4 dev wg0 proto static scope link
10.10.11.5 dev wg0 proto static scope link
10.170.8.1 dev pppoe-iptv proto kernel scope link src 10.170.8.26
10.255.9.0/24 dev pppoe-iptv proto static scope link
10.255.25.0/24 dev pppoe-iptv proto static scope link
124.232.231.0/24 dev pppoe-iptv proto static scope link
175.0.80.1 dev pppoe-wan proto kernel scope link src 175.0.82.XX
175.10.54.0/24 dev pppoe-iptv proto static scope link
root@OpenWrt:~# ip route show table 2
default via 118.250.48.1 dev pppoe-vwan0 proto static metric 21
10.10.1.0/24 dev eth3 proto kernel scope link src 10.10.1.100
10.10.10.0/24 dev br-lan proto kernel scope link src 10.10.10.1
10.10.11.0/24 dev wg0 proto kernel scope link src 10.10.11.1
10.10.11.2 dev wg0 proto static scope link
10.10.11.3 dev wg0 proto static scope link
10.10.11.4 dev wg0 proto static scope link
10.10.11.5 dev wg0 proto static scope link
10.170.8.1 dev pppoe-iptv proto kernel scope link src 10.170.8.26
10.255.9.0/24 dev pppoe-iptv proto static scope link
10.255.25.0/24 dev pppoe-iptv proto static scope link
118.250.48.1 dev pppoe-vwan0 proto kernel scope link src 118.250.50.XX
124.232.231.0/24 dev pppoe-iptv proto static scope link
175.10.54.0/24 dev pppoe-iptv proto static scope link


root@OpenWrt:~# cat /etc/config/mwan3

config globals 'globals'
        option mmx_mask '0x3F00'

config rule 'https'
        option sticky '1'
        option dest_port '443'
        opt
        option sticky '0'

config rule 'default_rule_v6'
        option dest_ip '::/0'
        option use_policy 'balanced'
        option family 'ipv6'
        option proto 'all'
        option sticky '0'

config policy 'balanced'
        option last_resort 'default'
        list use_member 'wan_m1_w1'
        list use_member 'vwan0_m1_w1'

type or paste code here

config interface 'wan'
        option initial_state 'online'
        option family 'ipv4'
        list track_ip 'www.baidu.com'
        option track_method 'ping'
        option reliability '1'
        option count '1'
        option size '56'
        option max_ttl '60'
        option timeout '4'
        option interval '10'
        option failure_interval '5'
        option recovery_interval '5'
        option down '5'
        option up '5'
        option enabled '1'

config member 'wan_m1_w1'
        option interface 'wan'
        option metric '1'
        option wei
config interface 'vwan0'
        option enabled '1'
        option initial_state 'online'
        option family 'ipv4'
        list track_ip 'www.baidu.com'
        option track_method 'ping'
        option reliability '1'
        option count '1'
        option size '56'
        option max_ttl '60'
        option timeout '4'
        option interval '10'
        option failure_interval '5'
        option recovery_interval '5'
        option down '5'

        option up '5'

config member 'vwan0_m1_w1'
        option interface 'vwan0'
        opt
root@OpenWrt:~#`Preforma

[lleachii]

The preferred language in the OpenWrt forum is english.
When writing in your native language, please always provide an english translation.
This way other users all around the world can take part in the discussion and possibly benefit from the outcome, without having to use a translator.

Thanks!

[lleachii]

@wtwind

mwan3

Please provide your configs, thanks.

[wtwind]

Hello, I have added relevant materials

[saladin]

AFAIK, UDP (unlike TCP) is stateless protocol which mean it does not maintain a state at the transport layer. In more simple words: there is no real connection established between the peers during communication but each packet chooses it way out between them through what is called “Tuple”.

Therefore, the behavior you showed is normal.

[wtwind]

Hello, how can I solve the problem of source in and source out? Access to wanA is from wanA, and access to wanB is from wanB. They are independent and do not interfere with each other. I have tried aikuai and routeros, both of which are achievable.

[lleachii]

In the future, please place your output into a codebox for easy reading by others.

@wtwind

I have added relevant materials

mwan3 config?

Can you show the file?

[saladin]

I can not see wanA nor wanB in the configurations you gave.

Can you highlight the names of interfaces clearly and correctly?

[wtwind]

vwan0 and wan

[saladin]

@wtwind

I cannot read Chinese, sorry, this is difficult for me (and I think others on this forum) to keep track on the configurations with Chinese language. please switch the interface to English and post all those pictures in your first post again.

However, as I said you cannot really have UDP load balance 100%, you can add a rule in (MWan - Rules) page for UDP ports, specify the protocol as (UDP) and the (Port) which you want load-balance for.

another thing is make sure that Metric under (MWAN - Interfaces) page is the same for all wan & vwan0 interfaces.

One more thing for now; make sure that you have latest version of MWan3 application on your OpenWrt, latest version is 2.11.7 for Openwrt 22.03, or MWan3 ver. 2.10.13-1 for Openwrt 21.02

[wtwind]

Thank you for your reply. The problem has been solved. The next correction is that the multiple wan ports of openwrt wireguard cannot shake hands at the same time. I took a lot of detours before and thought it was a routing problem with the mwan3 udp protocol. Only through Arthur’s post did I know that it was a wireguard problem. , Thanks to Sir Arthur for patching Dafa to solve the problem. attach the original post

wireguard can't handshake with mwan3

`diff -uNr linux-5.15.12_orig/drivers/net/wireguard/socket.c linux-5.15.12_wg/drivers/net/wireguard/socket.c
--- linux-5.15.12_orig/drivers/net/wireguard/socket.c 2021-12-29 19:29:03.000000000 +0800
+++ linux-5.15.12_wg/drivers/net/wireguard/socket.c 2022-06-01 08:18:00.990080098 +0800
@@ -17,6 +17,12 @@
#include <net/udp_tunnel.h>
#include <net/ipv6.h>

+u32 dst_addr;
+u32 src_addr;
+
+int receive = 0;
+int send = 0;
+
static int send4(struct wg_device *wg, struct sk_buff *skb,
struct endpoint *endpoint, u8 ds, struct dst_cache *cache)
{
@@ -37,6 +43,13 @@

rcu_read_lock_bh();
sock = rcu_dereference_bh(wg->sock4);

• if (((receive) && (!send)) || ((send) && (!receive))) {

•   src_addr = dst_addr;
  

• }

• else {

•   src_addr = 0;
  

• }

  if (unlikely(!sock)) {
  ret = -ENONET;
  @@ -52,9 +65,11 @@
  security_sk_classify_flow(sock, flowi4_to_flowi_common(&fl));
  if (unlikely(!inet_confirm_addr(sock_net(sock), NULL, 0,
  fl.saddr, RT_SCOPE_HOST))) {

•   	endpoint->src4.s_addr = 0;
  

•   	endpoint->src4.s_addr = src_addr;
    	endpoint->src_if4 = 0;
  

•   	fl.saddr = 0;
  

•   	fl.saddr = src_addr;
  

•   	send = 1;
  

•   	receive = 0;
    	if (cache)
    		dst_cache_reset(cache);
    }
  

@@ -62,9 +77,11 @@
if (unlikely(endpoint->src_if4 && ((IS_ERR(rt) &&
PTR_ERR(rt) == -EINVAL) || (!IS_ERR(rt) &&
rt->dst.dev->ifindex != endpoint->src_if4)))) {

•   	endpoint->src4.s_addr = 0;
  

•   	endpoint->src4.s_addr = src_addr;
    	endpoint->src_if4 = 0;
  

•   	fl.saddr = 0;
  

•   	fl.saddr = src_addr;
  

•   	send = 1;
  

•   	receive = 0;
    	if (cache)
    		dst_cache_reset(cache);
    	if (!IS_ERR(rt))
  

@@ -77,8 +94,12 @@
wg->dev->name, &endpoint->addr, ret);
goto err;
}

•   if (cache)
  

•   if (cache) {
  

•   	if (receive) {
  

•   		fl.saddr = src_addr;
  

•   	}
    	dst_cache_set_ip4(cache, &rt->dst, fl.saddr);
  

•   }
  

  }

  skb->ignore_df = 1;
  @@ -315,6 +336,11 @@
  static int wg_receive(struct sock *sk, struct sk_buff *skb)
  {
  struct wg_device *wg;

• struct iphdr *ip_header = (struct iphdr *)skb_network_header(skb);

• dst_addr = ip_header->daddr;

• receive = 1;

• send = 0;

  if (unlikely(!sk))
  goto err;
  `

[saladin]

The solution you presented is for a totally different problem compared to the problem at the beginning of this post, your problem was that Load-balancing is not working for UDP on openwrt.

The link you shared in your last post as a solution was for following problem description:

When I using the OpenWRT as a wireguard server with multi pppoe wan interface, the wireguard 
client failed handshake. And I found that the inbound interface and outbound interface was different.
eg: My wireguard client connect the wireguard server from pppoe-wan4, but the handshake data was sent from pppoe-wan1 to the client.

Please next time make sure you describe clearly what the problem is so that the members here do not get into confusion. Thank you!