I'm current setting up a portable router with 4G LTE module. I got a problem with the wireguard configuration and would be appreciated if anyone can help.
The issue is when I configured wireguard to connect to remote peer(also openwrt router) I can ping the wireguard interface address successfully but cannot ping its LAN network.
And the route table. The wwan0 is the WAN interface connect to the LTE module.
default dev wg0 proto static scope link
default via 10.212.73.117 dev wwan0 proto static src 10.212.73.116 metric 12
10.10.10.0/24 dev ipsec0 proto kernel scope link src 10.10.10.1
10.212.73.112/29 dev wwan0 proto static scope link metric 12
11.22.33.44 via 10.212.73.117 dev wwan0 proto static metric 12
192.168.5.0/24 dev br-lan proto kernel scope link src 192.168.5.1
192.168.66.0/24 dev wg0 proto kernel scope link src 192.168.66.5
Ping to peer's wireguard interface(192.168.66.1) can get response but not for the remote LAN network 192.168.0.0/24. And the output of "ip route get 192.168.0.1" looks normal. It matches the wg0 interface as expected.
root@oprouter1:~# ip route get 192.168.0.1
192.168.0.1 dev wg0 src 192.168.66.5 uid 0
cache
The outputs of "wg" command on both side are just as expected. And I'm pretty sure that the remote wireguard configuration has no problem because my another router with the exact same wireguard settings can connect to this remote router and access to its LAN network without any problem.
Also I got it work on this portable router once. I don't remembe what I did. It just works. It can access to remote LAN devices. But after I reboot the device and it does not work again. There's no setting change at all.
The remote firewall must allow routing from wg to its lan.
If the remote machine terminating the wireguard tunnel is not the main router in the remote lan, a route back to the wireguard tunnel and (if you do symmetric routing) the local lan must be installed in the remote lan's main router.
As configured the local router should be accessing the Internet via wireguard and using the remote router's ISP, is that working?
As you have put wg0 in the local router wan zone that will cause local LAN to be NATd into the wireguard tunnel as it leaves the local router. That will still work but is probably not what you want, since you're otherwise set up for symmetric routing. Place wg into the lan zone for testing, later-- if you want restrictions on VPN traffic-- you can make a separate zone with stricter rules.
On remote the wg interface is in the same zone as LAN. And below is the route table of remote router.
default via 11.22.33.1 dev pppoe-wan proto static
11.22.33.1 dev pppoe-wan proto kernel scope link src 11.22.33.44
192.168.0.0/24 dev br-lan.99 proto kernel scope link src 192.168.0.1
192.168.3.0/24 dev br-lan.3 proto kernel scope link src 192.168.3.1
192.168.5.0/24 dev wg0 proto static scope link
192.168.9.0/24 dev wg0 proto static scope link
192.168.11.0/24 dev wg0 proto static scope link
192.168.66.0/24 dev wg0 proto kernel scope link src 192.168.66.1
192.168.66.2 dev wg0 proto static scope link
192.168.66.3 dev wg0 proto static scope link
192.168.66.4 dev wg0 proto static scope link
192.168.66.5 dev wg0 proto static scope link
I just tried to put wg0 into the firewall zone lan and it makes no difference. Actually I put it in WAN because it is recommanded by document. See https://openwrt.org/docs/guide-user/services/vpn/wireguard/client.
Also on another router I mentioned it was also put in wan zone and it works perfectly.
There's another reason that I believe the problem is not with the remote wireguard setting. I ever tried to put a route for 192.168.0.0/24 through wg0 on local router and it just worked. See below.
root@oprouter1:~# ip route
default dev wg0 proto static scope link
default via 10.212.73.117 dev wwan0 proto static src 10.212.73.116 metric 12
10.10.10.0/24 dev ipsec0 proto kernel scope link src 10.10.10.1
10.212.73.112/29 dev wwan0 proto static scope link metric 12
11.22.33.44 via 10.212.73.117 dev wwan0 proto static metric 12
192.168.5.0/24 dev br-lan proto kernel scope link src 192.168.5.1
192.168.66.0/24 dev wg0 proto kernel scope link src 192.168.66.5
root@oprouter1:~#
root@oprouter1:~# ip route add 192.168.0.0/24 dev wg0
root@oprouter1:~#
root@oprouter1:~# ip route
default dev wg0 proto static scope link
default via 10.212.73.117 dev wwan0 proto static src 10.212.73.116 metric 12
10.10.10.0/24 dev ipsec0 proto kernel scope link src 10.10.10.1
10.212.73.112/29 dev wwan0 proto static scope link metric 12
11.22.33.44 via 10.212.73.117 dev wwan0 proto static metric 12
192.168.0.0/24 dev wg0 scope link
192.168.5.0/24 dev br-lan proto kernel scope link src 192.168.5.1
192.168.66.0/24 dev wg0 proto kernel scope link src 192.168.66.5
root@oprouter1:~#
root@oprouter1:~# ping 192.168.0.1
PING 192.168.0.1 (192.168.0.1): 56 data bytes
64 bytes from 192.168.0.1: seq=0 ttl=64 time=44.676 ms
64 bytes from 192.168.0.1: seq=1 ttl=64 time=42.964 ms
^C
--- 192.168.0.1 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 42.964/43.820/44.676 ms
root@oprouter1:~#
No it's not. I can access internet but the displayed public IP is still the one from local LTE module. So looks not only the 192.168.0.0/24 network but also all the destination(except wg interface) are still through the local WAN interface directly.
Another finding, when I do "ifdown wwan_lte" and then "ifup wwan_lte", in the first maybe 20 seconds ping 192.168.0.1 works. But later it gets no response.
root@oprouter1:~# ifdown wwan_lte
root@oprouter1:~# ifup wwan_lte
root@oprouter1:~# ping 192.168.0.1
PING 192.168.0.1 (192.168.0.1): 56 data bytes
64 bytes from 192.168.0.1: seq=0 ttl=64 time=21.282 ms
64 bytes from 192.168.0.1: seq=1 ttl=64 time=33.505 ms
64 bytes from 192.168.0.1: seq=2 ttl=64 time=54.755 ms
64 bytes from 192.168.0.1: seq=3 ttl=64 time=33.085 ms
64 bytes from 192.168.0.1: seq=4 ttl=64 time=19.777 ms
64 bytes from 192.168.0.1: seq=5 ttl=64 time=53.475 ms
64 bytes from 192.168.0.1: seq=6 ttl=64 time=53.188 ms
64 bytes from 192.168.0.1: seq=7 ttl=64 time=31.181 ms
64 bytes from 192.168.0.1: seq=8 ttl=64 time=52.639 ms
64 bytes from 192.168.0.1: seq=9 ttl=64 time=56.123 ms
64 bytes from 192.168.0.1: seq=10 ttl=64 time=70.884 ms
64 bytes from 192.168.0.1: seq=11 ttl=64 time=51.540 ms
64 bytes from 192.168.0.1: seq=12 ttl=64 time=42.127 ms
64 bytes from 192.168.0.1: seq=13 ttl=64 time=42.057 ms
64 bytes from 192.168.0.1: seq=14 ttl=64 time=51.144 ms
64 bytes from 192.168.0.1: seq=15 ttl=64 time=49.176 ms
64 bytes from 192.168.0.1: seq=16 ttl=64 time=47.269 ms
64 bytes from 192.168.0.1: seq=17 ttl=64 time=41.616 ms
64 bytes from 192.168.0.1: seq=18 ttl=64 time=41.239 ms
64 bytes from 192.168.0.1: seq=19 ttl=64 time=28.167 ms
64 bytes from 192.168.0.1: seq=20 ttl=64 time=50.011 ms
64 bytes from 192.168.0.1: seq=21 ttl=64 time=39.498 ms
64 bytes from 192.168.0.1: seq=22 ttl=64 time=39.129 ms
64 bytes from 192.168.0.1: seq=23 ttl=64 time=27.082 ms
^C
--- 192.168.0.1 ping statistics ---
1103 packets transmitted, 24 packets received, 97% packet loss
round-trip min/avg/max = 19.777/42.914/70.884 ms
root@oprouter1:~#
Finally I identify the cause. Right at the moment the ping stops getting response I see below log from logread output.
Wed Nov 22 15:07:44 2023 user.notice mwan3track[13375]: Interface wwan_lte (wwan0) is online
Wed Nov 22 15:07:45 2023 user.notice mwan3[14777]: Execute ifup event on interface wwan_lte (wwan0)
Wed Nov 22 15:07:45 2023 user.notice mwan3[14777]: Starting tracker on interface wwan_lte (wwan0)
I was not aware there's a load balancing package installed and running. Looks it is interferring with the wireguard route. After stopping the mwan3 service the ping just works. And all the networks behind remote peer became accessable.