Just a peer-to-peer network topology. but I am not able to capture the wireguard packets between them. I configured the firewall on both sides with 2x Masquerading and 1x Traffic Rules.
I am not sure about the meaning of "lan => wan (Masquerading)" but it sounds the VPN does not work without enabling "lan => wan (Masquerading)". Can someone tell me if my firewall configuration is correct?
So a bunch more context is required here, but generally speaking, masquerading must be disabled on the lan if this is your main router.
So...
And let's see some details about your configuration:
Please copy the output of the following commands and post it here using the "Preformatted text </> " button:
Remember to redact passwords, MAC addresses and any public IP addresses you may have:
ubus call system board
cat /etc/config/network
cat /etc/config/firewall
In a lan-lan use case you should not masquerade into the tunnel on either end. When the allowed_ips are set properly and route_allowed_ips is also enabled, symmetric routes to the other LAN through the tunnel will be established. (This does require the two LANs to be non-overlapping IP subnets.)
The wireguard interface can be in the lan zone or in a new zone which is allowed to forward to and from lan.
First of all, I really thanks for your help. My topology is rather complicated, with all nodes running openwrt + keepalived + wireguard. and local nodes having 2x WAN while remote nodes only have 1x WAN. I used keepalived to virtualize the WAN into VIP and I configured the wireguard endpoint to be the VIP on the other end.
Currently, for testing purposes, I have shut down the backups on both ends, leaving only the masters, resulting in the simplified topology diagram below.
My biggest problem is that without the " lan => wan (Masquerading) " rule, I still can ping 192.168.1.1 from 192.168.8.20, but I cannot ping 192.168.1.20 from 192.168.8.1 or 192.168.8.20, also same in reverse. When I tried to traceroute 192.168.1.20 from 192.168.8.20:
the first hop is the local gateway 192.168.8.1,
the second hop is the wireguard IP address 192.168.100.1 of the remote end
then it got stuck with unknow destination
This is strange to me, as I can see the existence of 192.168.1.20 in the ARP table on the remote end. It seems to be stuck at the wireguard IP(192.168.100.1) to the destination 192.168.1.20 on the remote end.
You don't want masquerading... make sure it is disabled on both sides (for the lan zone, as well as the zone WG is in, if it is not in the lan zone). The only zone that should have masquerading is the wan.
Then, install static routes on both sides...
If this doesn't solve the problem, please post your configs for both sides - we can review them and help you figure out what might be wrong.
Side note, but worth considering: Be aware that cable and dsl modems may actually use the 192.168.100.0/24 subnet -- every Arris cable modem I've used has held the address 192.168.100.1 for it's status/admin page (while still passing the public IP to the wan of my router). So the WG addresses you are using could possibly conflict there if you have a modem that uses that address.
Assuming that VPN is installed on each LAN's default gateway, you don't need static routes when route_allowed_ips and allowed_ips are properly specified:
https://openwrt.org/docs/guide-user/services/vpn/wireguard/extras#site-to-site
Yeah, that is true. I was thinking about it both ways, but I don’t know what else is happening on those routers - sometimes static routes can be useful, other times the allowed ips (which with the route allowed ips option m, achieves basically the same thing).
The main issue is that ping is not working on one side. I suspect that this is due to incorrect firewall configuration. Although the WireGuard tunnel already exists, all data is still being masqueraded to the WAN, which means it's not going through the WireGuard tunnel. In the following screenshot, I used the Wireshark tool to capture inbound packets on the Local Side (192.168.1.1), but I didn't see any WireGuard packets from the Remote Side (192.168.8.1 or 8.20), only ICMP requests that were unreachable.
I unchecked Masquerading in the ”lan => wan“ rule. The firewall configuration on both sides are same as following，but the problem still persists:
I assigned wg0 to the Lan zone.
I unchecked Masquerading in the lan => wan configuration.
I created a Traffic rule to allow wireguard traffic to this device.
PC A at the Remote Side:
ipconfig:
IPv4 Address. . . . . . . . . . . : 192.168.8.20
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . : 192.168.8.1
ping 192.168.1.1
Reply from 192.168.1.1: bytes=32 time=2ms TTL=63
Reply from 192.168.1.1: bytes=32 time=1ms TTL=63
ping 192.168.1.20 <------Unable to ping through 192.168.1.20 !!!!!
Request timed out.
Request timed out.
tracert 192.168.1.1
1 1 ms <1 ms <1 ms 192.168.8.1
2 2 ms 1 ms 1 ms 192.168.1.1
tracert 192.168.1.20：
1 <1 ms <1 ms <1 ms 192.168.8.1
2 2 ms 1 ms 1 ms 192.168.100.1
3 * * * Request timed out.
Remote Side's Openwrt:
ip route get 192.168.1.1：
192.168.1.1 dev wg0 src 192.168.100.2 uid 0
cache
ip route get 192.168.1.20：
192.168.1.20 dev wg0 src 192.168.100.2 uid 0
cache
wg:
interface: wg0
public key: 0Mjp6LkFoUGx4OFECV71MKqyeJOtbgHnTCQZXxOe8Sw=
private key: (hidden)
listening port: 51820
peer: JcZsifvDI9RyKT0AWeK7A0+AVYJhjFc81O1ZHj0DDng=
endpoint: 8.8.8.8:51820
allowed ips: 192.168.100.0/24, 192.168.1.0/24, 10.10.10.0/24 <----确认已经allow 192.168.1.0/24
latest handshake: 16 seconds ago
transfer: 9.41 MiB received, 8.70 MiB sent
persistent keepalive: every 10 seconds
route -n：
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 7.7.7.1 0.0.0.0 UG 0 0 0 wan
6.6.6.0 0.0.0.0 255.255.255.0 U 0 0 0 lan0
7.7.7.0 0.0.0.0 255.255.255.0 U 10 0 0 wan
8.8.8.2 7.7.7.1 255.255.255.255 UGH 10 0 0 wan
10.10.10.0 0.0.0.0 255.255.255.0 U 0 0 0 wg0
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 wg0
192.168.8.0 0.0.0.0 255.255.255.0 U 0 0 0 br-lan
192.168.100.0 0.0.0.0 255.255.255.0 U 0 0 0 wg0
ip route：
default via 7.7.7.1 dev wan proto static
6.6.6.0/24 dev lan0 scope link src 6.6.6.2
7.7.7.0/24 dev wan proto static scope link metric 10
8.8.8.2 via 7.7.7.1 dev wan proto static metric 10
10.10.10.0/24 dev wg0 proto static scope link
192.168.1.0/24 dev wg0 proto static scope link
192.168.8.0/24 dev br-lan proto kernel scope link src 192.168.8.1
192.168.100.0/24 dev wg0 proto static scope link
ip a:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 532
link/ether 94:83:c4:0a:42:98 brd ff:ff:ff:ff:ff:ff
inet6 fe80::9683:c4ff:fe0a:4298/64 scope link
valid_lft forever preferred_lft forever
3: wan@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 94:83:c4:0a:42:98 brd ff:ff:ff:ff:ff:ff
inet 7.7.7.8/24 brd 7.7.7.255 scope global wan
valid_lft forever preferred_lft forever
inet 7.7.7.2/24 scope global secondary wan
valid_lft forever preferred_lft forever
4: lan0@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 94:83:c4:0a:42:99 brd ff:ff:ff:ff:ff:ff
inet 6.6.6.8/24 brd 6.6.6.255 scope global lan0
valid_lft forever preferred_lft forever
inet 6.6.6.2/24 scope global secondary lan0
valid_lft forever preferred_lft forever
inet6 fe80::9683:c4ff:fe0a:4299/64 scope link
valid_lft forever preferred_lft forever
5: lan1@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-lan state UP group default qlen 1000
link/ether 94:83:c4:0a:42:99 brd ff:ff:ff:ff:ff:ff
7: teql0: <NOARP> mtu 1500 qdisc noop state DOWN group default qlen 100
link/void
8: usb0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel master br-lan state DOWN group default qlen 1000
link/ether 16:29:cd:27:d1:f8 brd ff:ff:ff:ff:ff:ff
51: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 94:83:c4:0a:42:99 brd ff:ff:ff:ff:ff:ff
inet 192.168.8.1/24 brd 192.168.8.255 scope global br-lan
valid_lft forever preferred_lft forever
67: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000
link/none
inet 192.168.100.2/32 brd 255.255.255.255 scope global wg0
valid_lft forever preferred_lft forever
PC B at the Local Side:
ipconfig：
IPv4 Address. . . . . . . . . . . : 192.168.1.20
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . : 192.168.1.1
ping 192.168.8.1
Reply from 192.168.8.1: bytes=32 time=1ms TTL=63
Reply from 192.168.8.1: bytes=32 time=2ms TTL=63
ping 192.168.8.20
Reply from 192.168.8.20: bytes=32 time=3ms TTL=126
Reply from 192.168.8.20: bytes=32 time=3ms TTL=126
tracert 192.168.8.1
1 <1 ms <1 ms <1 ms OpenWrt.lan [192.168.1.1]
2 2 ms 1 ms 1 ms 192.168.8.1
tracert 192.168.8.20
1 <1 ms <1 ms <1 ms OpenWrt.lan [192.168.1.1]
2 2 ms 2 ms 2 ms SHUANG2-SC01 [192.168.100.2]
3 3 ms 3 ms 3 ms SHUANG2-SC01 [192.168.8.20]
Local Side's Openwrt:
ip route get 192.168.8.1
192.168.8.1 dev wg0 src 192.168.100.1
ip route get 192.168.8.20
192.168.8.20 dev wg0 src 192.168.100.1
wg
interface: wg0
public key: JcZsifvDI9RyKT0AWeK7A0+AVYJhjFc81O1ZHj0DDng=
private key: (hidden)
listening port: 51820
peer: 0Mjp6LkFoUGx4OFECV71MKqyeJOtbgHnTCQZXxOe8Sw=
endpoint: 7.7.7.8:51820
allowed ips: 192.168.100.0/24, 192.168.8.0/24, 6.6.6.0/24
latest handshake: 1 minute, 34 seconds ago
transfer: 3.02 MiB received, 5.66 MiB sent
route -n：
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 8.8.8.1 0.0.0.0 UG 10 0 0 eth1
0.0.0.0 9.9.9.1 0.0.0.0 UG 20 0 0 eth2
6.6.6.0 0.0.0.0 255.255.255.0 U 0 0 0 wg0
8.8.8.0 0.0.0.0 255.255.255.0 U 10 0 0 eth1
9.9.9.0 0.0.0.0 255.255.255.0 U 20 0 0 eth2
10.10.10.0 0.0.0.0 255.255.255.0 U 0 0 0 eth3
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 br-lan
192.168.8.0 0.0.0.0 255.255.255.0 U 0 0 0 wg0
192.168.100.0 0.0.0.0 255.255.255.0 U 0 0 0 wg0
ip route：
default via 8.8.8.1 dev eth1 metric 10
default via 9.9.9.1 dev eth2 metric 20
6.6.6.0/24 dev wg0 scope link
8.8.8.0/24 dev eth1 scope link metric 10
9.9.9.0/24 dev eth2 scope link metric 20
10.10.10.0/24 dev eth3 scope link src 10.10.10.8
192.168.1.0/24 dev br-lan scope link src 192.168.1.1
192.168.8.0/24 dev wg0 scope link
192.168.100.0/24 dev wg0 scope link
ip a：
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br-lan state UP qlen 1000
link/ether 00:90:27:e7:17:01 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether 00:90:27:e7:17:02 brd ff:ff:ff:ff:ff:ff
inet 8.8.8.8/24 brd 8.8.8.255 scope global eth1
valid_lft forever preferred_lft forever
inet 8.8.8.2/24 scope global secondary eth1
valid_lft forever preferred_lft forever
inet6 fe80::290:27ff:fee7:1702/64 scope link
valid_lft forever preferred_lft forever
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether 00:90:27:e7:17:03 brd ff:ff:ff:ff:ff:ff
inet 9.9.9.8/24 brd 9.9.9.255 scope global eth2
valid_lft forever preferred_lft forever
inet 9.9.9.2/24 scope global secondary eth2
valid_lft forever preferred_lft forever
inet6 fe80::290:27ff:fee7:1703/64 scope link
valid_lft forever preferred_lft forever
5: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether 00:90:27:e7:17:04 brd ff:ff:ff:ff:ff:ff
inet 10.10.10.8/24 brd 10.10.10.255 scope global eth3
valid_lft forever preferred_lft forever
inet 10.10.10.2/24 scope global secondary eth3
valid_lft forever preferred_lft forever
inet6 fe80::290:27ff:fee7:1704/64 scope link
valid_lft forever preferred_lft forever
6: eth4: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master br-lan state DOWN qlen 1000
link/ether 00:90:27:e7:17:05 brd ff:ff:ff:ff:ff:ff
7: eth5: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether 00:90:27:e7:17:06 brd ff:ff:ff:ff:ff:ff
8: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
link/ether 00:90:27:e7:17:01 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.1/24 brd 192.168.1.255 scope global br-lan
valid_lft forever preferred_lft forever
inet6 fda6:e011:5fa9::1/60 scope global noprefixroute
valid_lft forever preferred_lft forever
inet6 fe80::290:27ff:fee7:1701/64 scope link
valid_lft forever preferred_lft forever
11: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN qlen 1000
link/[65534]
inet 192.168.100.1/32 brd 255.255.255.255 scope global wg0
valid_lft forever preferred_lft forever
******************** Testing: ********************
Local Side's OpenWrt: Adding "ip route add 192.168.1.20/32 via 192.168.100.1"
route -n：
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 8.8.8.1 0.0.0.0 UG 10 0 0 eth1
0.0.0.0 9.9.9.1 0.0.0.0 UG 20 0 0 eth2
6.6.6.0 0.0.0.0 255.255.255.0 U 0 0 0 wg0
8.8.8.0 0.0.0.0 255.255.255.0 U 10 0 0 eth1
9.9.9.0 0.0.0.0 255.255.255.0 U 20 0 0 eth2
10.10.10.0 0.0.0.0 255.255.255.0 U 0 0 0 eth3
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 br-lan
192.168.1.20 192.168.100.1 255.255.255.255 UGH 0 0 0 wg0
192.168.8.0 0.0.0.0 255.255.255.0 U 0 0 0 wg0
192.168.100.0 0.0.0.0 255.255.255.0 U 0 0 0 wg0
PC A at Remote Side:
ping 192.168.1.20
Pinging 192.168.1.20 with 32 bytes of data:
Reply from 192.168.100.1: Destination host unreachable.
Reply from 192.168.100.1: Destination host unreachable.
Reply from 192.168.100.1: Destination host unreachable.
Reply from 192.168.100.1: Destination host unreachable.
tracert 192.168.1.20
1 <1 ms <1 ms <1 ms 192.168.8.1
2 3 ms 2 ms 2 ms 192.168.100.1
3 192.168.100.1 reports: Destination host unreachable.
Let’s see the text configs on each side (please make it clear which is which)
Please copy the output of the following commands and post it here using the "Preformatted text </> " button:
Remember to redact passwords, MAC addresses and any public IP addresses you may have:
cat /etc/config/network
cat /etc/config/firewall
Thank you very much for your help！ Here is my configuration. I also included the "keepalived.conf" here, although it may not be useful at this moment, as I'm using its VIP as the endpoint.
Remote Side's Openwrt:
root@GL-MV1000:~# cat /etc/config/network
config interface 'loopback'
option ifname 'lo'
option proto 'static'
option ipaddr '127.0.0.1'
option netmask '255.0.0.0'
config globals 'globals'
option ula_prefix 'fdd4:f797:5643::/48'
config interface 'lan'
option type 'bridge'
option proto 'static'
option netmask '255.255.255.0'
option ip6assign '60'
option hostname 'GL-MV1000-298'
option ipaddr '192.168.8.1'
option ifname 'lan1 usb0'
config interface 'wan'
option ifname 'wan'
option ipv6 '0'
option proto 'static'
option netmask '255.255.255.0'
option ipaddr '7.7.7.8'
option gateway '7.7.7.1'
option metric '10'
config interface 'lan0'
option ifname 'lan0'
option macaddr '94:83:c4:0a:42:99'
config interface 'lan1'
option ifname 'lan1'
option macaddr '94:83:c4:0a:42:99'
config interface 'wg0'
option proto 'wireguard'
option listen_port '51820'
list addresses '192.168.100.2'
option private_key 'YEtZJtUPjmzEYa80tMbVSZdPWG7gge92ZnU2kpkd7WQ='
config wireguard_wg0
option route_allowed_ips '1'
option public_key 'JcZsifvDI9RyKT0AWeK7A0+AVYJhjFc81O1ZHj0DDng='
option endpoint_host '8.8.8.2'
option persistent_keepalive '10'
option endpoint_port '51820'
list allowed_ips '192.168.100.0/24'
list allowed_ips '192.168.1.0/24'
list allowed_ips '10.10.10.0/24'
config interface 'lanb'
option ifname 'lan0'
option proto 'static'
option netmask '255.255.255.0'
option ipaddr '6.6.6.8'
*************************************************************
root@GL-MV1000:~# cat /etc/config/firewall
config defaults
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'REJECT'
config zone
option name 'lan'
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'ACCEPT'
list network 'lan'
list network 'agg'
list network 'lanb'
list network 'wg0'
config zone
option name 'wan'
option output 'ACCEPT'
option masq '1'
option mtu_fix '1'
option input 'REJECT'
option forward 'REJECT'
list network 'wan'
config forwarding
option src 'lan'
option dest 'wan'
config rule
option name 'Allow-DHCP-Renew'
option src 'wan'
option proto 'udp'
option dest_port '68'
option target 'ACCEPT'
option family 'ipv4'
config rule
option name 'Allow-Ping'
option src 'wan'
option proto 'icmp'
option icmp_type 'echo-request'
option family 'ipv4'
option target 'ACCEPT'
config rule
option name 'Allow-IGMP'
option src 'wan'
option proto 'igmp'
option family 'ipv4'
option target 'ACCEPT'
config rule
option name 'Allow-DHCPv6'
option src 'wan'
option proto 'udp'
option dest_port '546'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-MLD'
option src 'wan'
option proto 'icmp'
option src_ip 'fe80::/10'
list icmp_type '130/0'
list icmp_type '131/0'
list icmp_type '132/0'
list icmp_type '143/0'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-ICMPv6-Input'
option src 'wan'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
list icmp_type 'router-solicitation'
list icmp_type 'neighbour-solicitation'
list icmp_type 'router-advertisement'
list icmp_type 'neighbour-advertisement'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-ICMPv6-Forward'
option src 'wan'
option dest '*'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-IPSec-ESP'
option src 'wan'
option dest 'lan'
option proto 'esp'
option target 'ACCEPT'
config rule
option name 'Allow-ISAKMP'
option src 'wan'
option dest 'lan'
option dest_port '500'
option proto 'udp'
option target 'ACCEPT'
config rule
option name 'wg'
list proto 'udp'
option src 'wan'
option dest_port '51820'
option target 'ACCEPT'
*************************************************************
root@GL-MV1000:~# cat /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
router_id HUAWEI1
script_user root
}
vrrp_instance WAN {
interface wan
state MASTER
virtual_router_id 51
priority 101
authentication {
auth_type PASS
auth_pass 1111
}
unicast_src_ip 7.7.7.8
unicast_peer {
7.7.7.7
}
virtual_ipaddress {
7.7.7.2/24 dev wan
}
preempt_delay 10
track_interface {
lan0
}
}
vrrp_instance LANb {
interface lan0
state MASTER
virtual_router_id 52
priority 101
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
6.6.6.2/24 dev lan0
}
preempt_delay 10
track_interface {
wan
}
notify_master "/bin/sh /etc/keepalived/wg_master.sh"
notify_backup "/bin/sh /etc/keepalived/wg_backup.sh"
notify_fault "/bin/sh /etc/keepalived/wg_backup.sh"
}
Local Side's Openwrt:
root@OpenWrt:~# cat /etc/config/network
config interface 'loopback'
option device 'lo'
option proto 'static'
option ipaddr '127.0.0.1'
option netmask '255.0.0.0'
config globals 'globals'
option ula_prefix 'fda6:e011:5fa9::/48'
config device
option name 'br-lan'
option type 'bridge'
list ports 'eth0'
list ports 'eth4'
config interface 'lan'
option device 'br-lan'
option proto 'static'
option ipaddr '192.168.1.1'
option netmask '255.255.255.0'
option ip6assign '60'
config interface 'wan'
option device 'eth1'
option proto 'static'
option ipaddr '8.8.8.8'
option netmask '255.255.255.0'
option gateway '8.8.8.1'
option metric '10'
config interface 'wanb'
option proto 'static'
option device 'eth2'
option ipaddr '9.9.9.8'
option netmask '255.255.255.0'
option gateway '9.9.9.1'
option metric '20'
config interface 'lanb'
option proto 'static'
option device 'eth3'
option ipaddr '10.10.10.8'
option netmask '255.255.255.0'
option gateway '8.8.8.1'
config interface 'wg0'
option proto 'wireguard'
option private_key 'GGv/4oUSIBdkUpt6231o/QObwtZ4Rpio9Kbh/Mm/c38='
option listen_port '51820'
list addresses '192.168.100.1'
config wireguard_wg0
option public_key '0Mjp6LkFoUGx4OFECV71MKqyeJOtbgHnTCQZXxOe8Sw='
option route_allowed_ips '1'
option endpoint_port '51820'
option endpoint_host '7.7.7.2'
list allowed_ips '192.168.100.0/24'
list allowed_ips '192.168.8.0/24'
list allowed_ips '6.6.6.0/24'
config device
option name 'eth3'
option acceptlocal '1'
*************************************************************
root@OpenWrt:~# cat /etc/config/firewall
config defaults
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'REJECT'
config zone
option name 'lan'
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'ACCEPT'
list network 'lan'
list network 'lanb'
list network 'wg0'
config zone
option name 'wan'
option output 'ACCEPT'
option mtu_fix '1'
option masq '1'
option input 'REJECT'
option forward 'REJECT'
list network 'wan'
list network 'wanb'
config forwarding
option src 'lan'
option dest 'wan'
config rule
option name 'Allow-DHCP-Renew'
option src 'wan'
option proto 'udp'
option dest_port '68'
option target 'ACCEPT'
option family 'ipv4'
config rule
option name 'Allow-Ping'
option src 'wan'
option proto 'icmp'
option icmp_type 'echo-request'
option family 'ipv4'
option target 'ACCEPT'
config rule
option name 'Allow-IGMP'
option src 'wan'
option proto 'igmp'
option family 'ipv4'
option target 'ACCEPT'
config rule
option name 'Allow-DHCPv6'
option src 'wan'
option proto 'udp'
option dest_port '546'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-MLD'
option src 'wan'
option proto 'icmp'
option src_ip 'fe80::/10'
list icmp_type '130/0'
list icmp_type '131/0'
list icmp_type '132/0'
list icmp_type '143/0'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-ICMPv6-Input'
option src 'wan'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
list icmp_type 'router-solicitation'
list icmp_type 'neighbour-solicitation'
list icmp_type 'router-advertisement'
list icmp_type 'neighbour-advertisement'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-ICMPv6-Forward'
option src 'wan'
option dest '*'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-IPSec-ESP'
option src 'wan'
option dest 'lan'
option proto 'esp'
option target 'ACCEPT'
config rule
option name 'Allow-ISAKMP'
option src 'wan'
option dest 'lan'
option dest_port '500'
option proto 'udp'
option target 'ACCEPT'
config rule
config rule
option name 'wg0'
list proto 'udp'
option src 'wan'
option dest_port '51820'
option target 'ACCEPT'
*************************************************************
root@OpenWrt:~# cat /etc/config/keepalived
config globals 'globals'
root@OpenWrt:~# cat /etc/config/keepalived.conf
cat: can't open '/etc/config/keepalived.conf': No such file or directory
root@OpenWrt:~# cat /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
router_id HUAWEI1
script_user root
}
vrrp_script check_wan {
script "/bin/sh /etc/keepalived/ping_sh.sh"
interval 20
fall 1
rise 1
weight -10
}
vrrp_instance WANa {
interface eth1
state MASTER
virtual_router_id 51
priority 101
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
8.8.8.2/24 dev eth1
}
track_script {
check_wan
}
preempt_delay 10
track_interface {
eth2
eth3
}
}
vrrp_instance WANb {
interface eth2
state MASTER
virtual_router_id 52
priority 101
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
9.9.9.2/24 dev eth2
}
track_script {
check_wan
}
preempt_delay 10
track_interface {
eth1
eth3
}
}
vrrp_instance LANb {
interface eth3
state MASTER
virtual_router_id 53
priority 101
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.10.10.2/24 dev eth3
}
track_script {
check_wan
}
preempt_delay 10
track_interface {
eth1
eth2
}
notify_master "/bin/sh /etc/keepalived/wg_master.sh"
notify_backup "/bin/sh /etc/keepalived/wg_backup.sh"
notify_fault "/bin/sh /etc/keepalived/wg_backup.sh"
}
It looks like you're using the gl-inet stock firmware on one of your devices... The usual advice is this:
It appears you are using firmware that is not from the official OpenWrt project.
When using forks/offshoots/vendor-specific builds that are "based on OpenWrt", there may be many differences compared to the official versions (hosted by OpenWrt.org). Some of these customizations may fundamentally change the way that OpenWrt works. You might need help from people with specific/specialized knowledge about the firmware you are using, so it is possible that advice you get here may not be useful.
You may find that the best options are:
If you believe that this specific issue is common to generic/official OpenWrt and/or the maintainers of your build have indicated as such, please feel free to clarify.
I'd personally recommend that you upgrade that unit to official OpenWrt. However, if you want to keep the stock firmware, there is a gl-inet employee on the forums that we can tag into this thread, if you'd like.
I'll take a look at the other side and see if I can spot any issues.
Thank you very much for your suggestion. Indeed, I am using a GL-Net on the remote side, and I will replace it with official Openwrt as soon as possible and then test it again. I will update to you once I have done so.
You've over-redacted your network files -- I can't really figure out what's going on.
RFC1918 addresses should not be redacted (anything in the 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16 is not public, and therefore safe to share). And the wan addresses should absolutely be redacted if they are public, but if they're RFC1918, it is even more important for us to see those.
I am currently in a lab environment where I use a switch to simulate the internet, using different VLANs to isolate or route between them. And I can use any IP for testing. But I can confirm that there is no routing between 192.168.8.0/24 and 192.168.1.0/24. Both subnets have to go through wg0 and then be masqueraded to the WAN port.
I see... so are the IP addresses you've shown the actual IP addresses you are using in this lab environment?
I would recommend using RFC1918 addresses, even in a lab environment, unless there is a specific reason to use what would be public IPs if this lab were actually connected ot the internet. This way it is less confusing (after all, you had to explain your IP address rationale), and it would also be able to be directly deployed to a live (internet connected) network.
Meanwhile, now I'm looking at the addresses on the wan of these routers... you have one side with 7.7.7.0/24 and the other side with 8.8.8.0/24 and 9.9.9.0/24. While I get that these are in an airgapped lab environment, you said you have a switch between these devices... you'll need a router to get traffic to flow between these two device's wan ports... where is the router in the equation?
Yes, I have a layer 3 switch to simulate an internet environment. The Remote side uses the 7.7.7.0/24 as its WAN, and the Local side uses the 8.8.8.0/24 and 9.9.9.0/24 as its two WANs. The switch in between allows them to communicate with each other.
Additionally, there are 2x LANs behind the Remote side using the 6.6.6.0/24 and 192.168.8.0/24 networks, and two LANs behind the Local side using the 10.10.10.0/24 and 192.168.1.0/24 networks. There is no routing set up for traffic between the LANs on the two sides.
I replaced the GL-Net router today, and now both sides are using OpenWrt 22.03.3. However, it seems like the problem is still the same as yesterday. I can ping the local ports on each peer side, but I cannot ping the PC that is directly connected to the local port.
PC_A(192.168.8.20) at the Remote Side:
>Ping 192.168.1.1
Reply from 192.168.1.1: bytes=32 time=1ms TTL=63
Reply from 192.168.1.1: bytes=32 time=1ms TTL=63
>ping 192.168.1.20
Request timed out. <------Unable to ping !
Request timed out.
>tracert 192.168.1.1
1 <1 ms <1 ms <1 ms OpenWrt_B_RemoteSide.lan [192.168.8.1]
2 1 ms 1 ms 1 ms 192.168.1.1
>tracert 192.168.1.20
1 <1 ms <1 ms <1 ms OpenWrt_B_RemoteSide.lan [192.168.8.1]
2 1 ms 1 ms 1 ms 192.168.100.1
3 * * * Request timed out.
PC_B(192.168.1.20) at the Local Side:
>ping 192.168.8.1
Reply from 192.168.8.1: bytes=32 time=1ms TTL=63
Reply from 192.168.8.1: bytes=32 time=1ms TTL=63
>ping 192.168.8.20
Request timed out. <------Unable to ping !
Request timed out.
>tracert 192.168.8.1
1 <1 ms <1 ms <1 ms OpenWrt_A_LocalSide.lan [192.168.1.1]
2 1 ms 1 ms 1 ms 192.168.8.1
>tracert 192.168.8.20
1 <1 ms <1 ms <1 ms OpenWrt_A_LocalSide.lan [192.168.1.1]
2 1 ms 1 ms 1 ms 192.168.100.2
3 * * * Request timed out.
Below are the firewall configurations on both sides, which are the same as yesterday's
Remote Side's Openwrt:
cat /etc/config/network
config interface 'loopback'
option device 'lo'
option proto 'static'
option ipaddr '127.0.0.1'
option netmask '255.0.0.0'
config globals 'globals'
option ula_prefix 'fd15:619b:2d98::/48'
config device
option name 'br-lan'
option type 'bridge'
list ports 'eth0'
config interface 'lan'
option device 'br-lan'
option proto 'static'
option netmask '255.255.255.0'
option ip6assign '60'
option ipaddr '192.168.8.1'
config interface 'wan'
option device 'eth1'
option proto 'static'
option ipaddr '7.7.7.8'
option netmask '255.255.255.0'
option gateway '7.7.7.1'
config interface 'lanb'
option proto 'static'
option device 'eth2'
option ipaddr '6.6.6.8'
option netmask '255.255.255.0'
config interface 'wg0'
option proto 'wireguard'
option private_key 'YEtZJtUPjmzEYa80tMbVSZdPWG7gge92ZnU2kpkd7WQ='
option listen_port '51820'
list addresses '192.168.100.2'
config wireguard_wg0
option public_key 'JcZsifvDI9RyKT0AWeK7A0+AVYJhjFc81O1ZHj0DDng='
list allowed_ips '192.168.100.0/24'
list allowed_ips '192.168.1.0/24'
list allowed_ips '10.10.10.0/24'
option route_allowed_ips '1'
option endpoint_port '51820'
option persistent_keepalive '25'
option endpoint_host '8.8.8.8'
********************************
cat /etc/config/firewall
config defaults
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'REJECT'
option synflood_protect '1'
config zone
option name 'lan'
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'ACCEPT'
list network 'lan'
list network 'lanb'
list network 'wg0'
config zone
option name 'wan'
option input 'REJECT'
option output 'ACCEPT'
option forward 'REJECT'
option masq '1'
option mtu_fix '1'
list network 'wan'
config forwarding
option src 'lan'
option dest 'wan'
config rule
option name 'Allow-DHCP-Renew'
option src 'wan'
option proto 'udp'
option dest_port '68'
option target 'ACCEPT'
option family 'ipv4'
config rule
option name 'Allow-Ping'
option src 'wan'
option proto 'icmp'
option icmp_type 'echo-request'
option family 'ipv4'
option target 'ACCEPT'
config rule
option name 'Allow-IGMP'
option src 'wan'
option proto 'igmp'
option family 'ipv4'
option target 'ACCEPT'
config rule
option name 'Allow-DHCPv6'
option src 'wan'
option proto 'udp'
option dest_port '546'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-MLD'
option src 'wan'
option proto 'icmp'
option src_ip 'fe80::/10'
list icmp_type '130/0'
list icmp_type '131/0'
list icmp_type '132/0'
list icmp_type '143/0'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-ICMPv6-Input'
option src 'wan'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
list icmp_type 'router-solicitation'
list icmp_type 'neighbour-solicitation'
list icmp_type 'router-advertisement'
list icmp_type 'neighbour-advertisement'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-ICMPv6-Forward'
option src 'wan'
option dest '*'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-IPSec-ESP'
option src 'wan'
option dest 'lan'
option proto 'esp'
option target 'ACCEPT'
config rule
option name 'Allow-ISAKMP'
option src 'wan'
option dest 'lan'
option dest_port '500'
option proto 'udp'
option target 'ACCEPT'
config rule
option name 'wg0'
list proto 'udp'
option src 'wan'
option dest_port '51820'
option target 'ACCEPT'
Local Side's Openwrt:
cat /etc/config/network
config interface 'loopback'
option device 'lo'
option proto 'static'
option ipaddr '127.0.0.1'
option netmask '255.0.0.0'
config globals 'globals'
option ula_prefix 'fda6:e011:5fa9::/48'
config device
option name 'br-lan'
option type 'bridge'
list ports 'eth0'
list ports 'eth4'
config interface 'lan'
option device 'br-lan'
option proto 'static'
option ipaddr '192.168.1.1'
option netmask '255.255.255.0'
option ip6assign '60'
config interface 'wan'
option device 'eth1'
option proto 'static'
option ipaddr '8.8.8.8'
option netmask '255.255.255.0'
option gateway '8.8.8.1'
option metric '10'
config interface 'wanb'
option proto 'static'
option device 'eth2'
option ipaddr '9.9.9.8'
option netmask '255.255.255.0'
option gateway '9.9.9.1'
option metric '20'
config interface 'lanb'
option proto 'static'
option device 'eth3'
option ipaddr '10.10.10.8'
option netmask '255.255.255.0'
option gateway '8.8.8.1'
config interface 'wg0'
option proto 'wireguard'
option private_key 'GGv/4oUSIBdkUpt6231o/QObwtZ4Rpio9Kbh/Mm/c38='
option listen_port '51820'
list addresses '192.168.100.1'
config wireguard_wg0
option public_key '0Mjp6LkFoUGx4OFECV71MKqyeJOtbgHnTCQZXxOe8Sw='
option route_allowed_ips '1'
option endpoint_port '51820'
list allowed_ips '192.168.100.0/24'
list allowed_ips '192.168.8.0/24'
list allowed_ips '6.6.6.0/24'
option endpoint_host '7.7.7.8'
option persistent_keepalive '25'
config device
option name 'eth3'
option acceptlocal '1'
********************************
cat /etc/config/firewall
config defaults
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'REJECT'
config zone
option name 'lan'
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'ACCEPT'
list network 'lan'
list network 'lanb'
list network 'wg0'
config zone
option name 'wan'
option output 'ACCEPT'
option mtu_fix '1'
option masq '1'
option input 'REJECT'
option forward 'REJECT'
list network 'wan'
list network 'wanb'
config forwarding
option src 'lan'
option dest 'wan'
config rule
option name 'Allow-DHCP-Renew'
option src 'wan'
option proto 'udp'
option dest_port '68'
option target 'ACCEPT'
option family 'ipv4'
config rule
option name 'Allow-Ping'
option src 'wan'
option proto 'icmp'
option icmp_type 'echo-request'
option family 'ipv4'
option target 'ACCEPT'
config rule
option name 'Allow-IGMP'
option src 'wan'
option proto 'igmp'
option family 'ipv4'
option target 'ACCEPT'
config rule
option name 'Allow-DHCPv6'
option src 'wan'
option proto 'udp'
option dest_port '546'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-MLD'
option src 'wan'
option proto 'icmp'
option src_ip 'fe80::/10'
list icmp_type '130/0'
list icmp_type '131/0'
list icmp_type '132/0'
list icmp_type '143/0'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-ICMPv6-Input'
option src 'wan'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
list icmp_type 'router-solicitation'
list icmp_type 'neighbour-solicitation'
list icmp_type 'router-advertisement'
list icmp_type 'neighbour-advertisement'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-ICMPv6-Forward'
option src 'wan'
option dest '*'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-IPSec-ESP'
option src 'wan'
option dest 'lan'
option proto 'esp'
option target 'ACCEPT'
config rule
option name 'Allow-ISAKMP'
option src 'wan'
option dest 'lan'
option dest_port '500'
option proto 'udp'
option target 'ACCEPT'
config rule
config rule
option name 'wg0'
list proto 'udp'
option src 'wan'
option dest_port '51820'
option target 'ACCEP
I tried a different firewall configuration， but the problem still persists. Here are the firewall configurations, I deleted the wg0 traffic rule shown above.
PC_A(192.168.8.20) at the Remote Side:
>ping 192.168.1.1
Reply from 192.168.1.1: bytes=32 time=1ms TTL=63
Reply from 192.168.1.1: bytes=32 time=1ms TTL=63
>ping 192.168.1.20
Reply from 192.168.100.1: Destination port unreachable.
Reply from 192.168.100.1: Destination port unreachable.
>tracert 192.168.1.1
1 <1 ms <1 ms <1 ms OpenWrt_B_RemoteSide.lan [192.168.8.1]
2 1 ms 1 ms 1 ms 192.168.1.1
>tracert 192.168.1.20
1 <1 ms <1 ms <1 ms OpenWrt_B_RemoteSide.lan [192.168.8.1]
2 1 ms <1 ms 1 ms 192.168.100.1
3 192.168.100.1 reports: Destination protocol unreachable.
PC_B(192.168.1.20) at the Local Side:
>ping 192.168.8.1
Reply from 192.168.8.1: bytes=32 time=1ms TTL=63
Reply from 192.168.8.1: bytes=32 time=1ms TTL=63
>ping 192.168.8.20
Reply from 192.168.100.2: Destination port unreachable.
Reply from 192.168.100.2: Destination port unreachable.
>tracert 192.168.8.1
1 <1 ms <1 ms <1 ms OpenWrt_A_LocalSide.lan [192.168.1.1]
2 1 ms 1 ms <1 ms 192.168.8.1
>tracert 192.168.8.20
1 <1 ms <1 ms <1 ms OpenWrt_A_LocalSide.lan [192.168.1.1]
2 1 ms 1 ms 1 ms 192.168.100.2
3 192.168.100.2 reports: Destination protocol unreachable.
What is this host? Mac? Windows? Linux? Something else?
192.168.1.20 and 192.168.8.20 both are windows 10
Check the windows firewall. There is a high probability that the problem is there. By default, Windows firewall will not accept connections from a different subnet. You need to modify the firewall to allow this. The simple way to test this is to simply turn off the windows firewall and run the test.
Thank you for reminding me. I can confirm that the firewall has been shut down on both PCs. During today's testing, there was some progress. I discovered that I had mistakenly created a gateway with the IP address of 6.6.6.1 on the switch. Although I did not use this network for testing, the gateway was included in the LAN zone on the firewall, which can caused some issues.
PC_A(192.168.8.20) at the Remote Side:
>tracert 192.168.1.1
1 <1 ms <1 ms <1 ms OpenWrt_B_RemoteSide.lan [192.168.8.1]
2 1 ms 1 ms 1 ms 192.168.1.1
>tracert 192.168.1.20
1 <1 ms <1 ms <1 ms OpenWrt_B_RemoteSide.lan [192.168.8.1]
2 1 ms 1 ms <1 ms 192.168.100.1
3 1 ms 1 ms 1 ms LOANERLAPTOP-SC [192.168.1.20]
Remote Side's Openwrt:
# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 7.7.7.1 0.0.0.0 UG 0 0 0 eth1
6.6.6.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2
7.7.7.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
8.8.8.2 7.7.7.1 255.255.255.255 UGH 0 0 0 eth1
8.8.8.8 7.7.7.1 255.255.255.255 UGH 0 0 0 eth1
10.10.10.0 0.0.0.0 255.255.255.0 U 0 0 0 wg0
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 wg0
192.168.8.0 0.0.0.0 255.255.255.0 U 0 0 0 br-lan
192.168.100.0 0.0.0.0 255.255.255.0 U 0 0 0 wg0
# ip route
default via 7.7.7.1 dev eth1
6.6.6.0/24 dev eth2 scope link src 6.6.6.8
7.7.7.0/24 dev eth1 scope link src 7.7.7.8
8.8.8.2 via 7.7.7.1 dev eth1
8.8.8.8 via 7.7.7.1 dev eth1
10.10.10.0/24 dev wg0 scope link
192.168.1.0/24 dev wg0 scope link
192.168.8.0/24 dev br-lan scope link src 192.168.8.1
192.168.100.0/24 dev wg0 scope link
# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br-lan state UP qlen 1000
link/ether 00:f1:f3:1f:da:24 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether 00:f1:f3:1f:da:25 brd ff:ff:ff:ff:ff:ff
inet 7.7.7.8/24 brd 7.7.7.255 scope global eth1
valid_lft forever preferred_lft forever
inet 7.7.7.2/24 scope global secondary eth1
valid_lft forever preferred_lft forever
inet6 fe80::2f1:f3ff:fe1f:da25/64 scope link
valid_lft forever preferred_lft forever
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether 00:f1:f3:1f:da:26 brd ff:ff:ff:ff:ff:ff
inet 6.6.6.8/24 brd 6.6.6.255 scope global eth2
valid_lft forever preferred_lft forever
inet 6.6.6.2/24 scope global secondary eth2
valid_lft forever preferred_lft forever
inet6 fe80::2f1:f3ff:fe1f:da26/64 scope link
valid_lft forever preferred_lft forever
5: eth3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether 00:f1:f3:1f:da:27 brd ff:ff:ff:ff:ff:ff
6: eth4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether 00:f1:f3:1f:da:28 brd ff:ff:ff:ff:ff:ff
7: eth5: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether 00:f1:f3:1f:da:29 brd ff:ff:ff:ff:ff:ff
8: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
link/ether 00:f1:f3:1f:da:24 brd ff:ff:ff:ff:ff:ff
inet 192.168.8.1/24 brd 192.168.8.255 scope global br-lan
valid_lft forever preferred_lft forever
inet6 fd15:619b:2d98::1/60 scope global noprefixroute
valid_lft forever preferred_lft forever
inet6 fe80::2f1:f3ff:fe1f:da24/64 scope link
valid_lft forever preferred_lft forever
13: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN qlen 1000
link/[65534]
inet 192.168.100.2/32 brd 255.255.255.255 scope global wg0
valid_lft forever preferred_lft forever
# cat /etc/config/network
config interface 'loopback'
option device 'lo'
option proto 'static'
option ipaddr '127.0.0.1'
option netmask '255.0.0.0'
config globals 'globals'
option ula_prefix 'fd15:619b:2d98::/48'
config device
option name 'br-lan'
option type 'bridge'
list ports 'eth0'
config interface 'lan'
option device 'br-lan'
option proto 'static'
option netmask '255.255.255.0'
option ip6assign '60'
option ipaddr '192.168.8.1'
config interface 'wan'
option device 'eth1'
option proto 'static'
option ipaddr '7.7.7.8'
option netmask '255.255.255.0'
option gateway '7.7.7.1'
config interface 'lanb'
option proto 'static'
option device 'eth2'
option ipaddr '6.6.6.8'
option netmask '255.255.255.0'
config interface 'wg0'
option proto 'wireguard'
option private_key 'YEtZJtUPjmzEYa80tMbVSZdPWG7gge92ZnU2kpkd7WQ='
option listen_port '51820'
list addresses '192.168.100.2'
option defaultroute '0'
config wireguard_wg0
option public_key 'JcZsifvDI9RyKT0AWeK7A0+AVYJhjFc81O1ZHj0DDng='
list allowed_ips '192.168.100.0/24'
list allowed_ips '192.168.1.0/24'
list allowed_ips '10.10.10.0/24'
option route_allowed_ips '1'
option endpoint_port '51820'
option persistent_keepalive '25'
option endpoint_host '8.8.8.8'
************************************
# cat /etc/config/firewall
config defaults
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'REJECT'
option synflood_protect '1'
config zone
option name 'lan'
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'ACCEPT'
list network 'lan'
list network 'lanb'
list network 'wg0'
config zone
option name 'wan'
option input 'REJECT'
option output 'ACCEPT'
option forward 'REJECT'
option masq '1'
option mtu_fix '1'
list network 'wan'
config forwarding
option src 'lan'
option dest 'wan'
config rule
option name 'Allow-DHCP-Renew'
option src 'wan'
option proto 'udp'
option dest_port '68'
option target 'ACCEPT'
option family 'ipv4'
config rule
option name 'Allow-Ping'
option src 'wan'
option proto 'icmp'
option icmp_type 'echo-request'
option family 'ipv4'
option target 'ACCEPT'
config rule
option name 'Allow-IGMP'
option src 'wan'
option proto 'igmp'
option family 'ipv4'
option target 'ACCEPT'
config rule
option name 'Allow-DHCPv6'
option src 'wan'
option proto 'udp'
option dest_port '546'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-MLD'
option src 'wan'
option proto 'icmp'
option src_ip 'fe80::/10'
list icmp_type '130/0'
list icmp_type '131/0'
list icmp_type '132/0'
list icmp_type '143/0'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-ICMPv6-Input'
option src 'wan'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
list icmp_type 'router-solicitation'
list icmp_type 'neighbour-solicitation'
list icmp_type 'router-advertisement'
list icmp_type 'neighbour-advertisement'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-ICMPv6-Forward'
option src 'wan'
option dest '*'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-IPSec-ESP'
option src 'wan'
option dest 'lan'
option proto 'esp'
option target 'ACCEPT'
config rule
option name 'Allow-ISAKMP'
option src 'wan'
option dest 'lan'
option dest_port '500'
option proto 'udp'
option target 'ACCEPT'
config rule
option name 'wg0'
list proto 'udp'
option src 'wan'
option dest_port '51820'
option target 'ACCEPT'
PC_B(192.168.1.20) at the Local Side:
>tracert 192.168.8.1
1 <1 ms <1 ms <1 ms OpenWrt_A_LocalSide.lan [192.168.1.1]
2 1 ms <1 ms 1 ms 192.168.8.1
>tracert 192.168.8.20
1 <1 ms <1 ms <1 ms OpenWrt_A_LocalSide.lan [192.168.1.1]
2 1 ms 1 ms 1 ms 192.168.100.2
3 1 ms 1 ms 1 ms SHUANG2-SC01 [192.168.8.20]
Local Side's Openwrt:
# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 8.8.8.1 0.0.0.0 UG 10 0 0 eth1
0.0.0.0 9.9.9.1 0.0.0.0 UG 20 0 0 eth2
6.6.6.0 0.0.0.0 255.255.255.0 U 0 0 0 wg0
8.8.8.0 0.0.0.0 255.255.255.0 U 10 0 0 eth1
9.9.9.0 0.0.0.0 255.255.255.0 U 20 0 0 eth2
10.10.10.0 0.0.0.0 255.255.255.0 U 0 0 0 eth3
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 br-lan
192.168.8.0 0.0.0.0 255.255.255.0 U 0 0 0 wg0
192.168.100.0 0.0.0.0 255.255.255.0 U 0 0 0 wg0
# ip route
default via 8.8.8.1 dev eth1 metric 10
default via 9.9.9.1 dev eth2 metric 20
6.6.6.0/24 dev wg0 scope link
8.8.8.0/24 dev eth1 scope link metric 10
9.9.9.0/24 dev eth2 scope link metric 20
10.10.10.0/24 dev eth3 scope link src 10.10.10.8
192.168.1.0/24 dev br-lan scope link src 192.168.1.1
192.168.8.0/24 dev wg0 scope link
192.168.100.0/24 dev wg0 scope link
# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br-lan state UP qlen 1000
link/ether 00:90:27:e7:17:01 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether 00:90:27:e7:17:02 brd ff:ff:ff:ff:ff:ff
inet 8.8.8.8/24 brd 8.8.8.255 scope global eth1
valid_lft forever preferred_lft forever
inet 8.8.8.2/24 scope global secondary eth1
valid_lft forever preferred_lft forever
inet6 fe80::290:27ff:fee7:1702/64 scope link
valid_lft forever preferred_lft forever
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether 00:90:27:e7:17:03 brd ff:ff:ff:ff:ff:ff
inet 9.9.9.8/24 brd 9.9.9.255 scope global eth2
valid_lft forever preferred_lft forever
inet 9.9.9.2/24 scope global secondary eth2
valid_lft forever preferred_lft forever
inet6 fe80::290:27ff:fee7:1703/64 scope link
valid_lft forever preferred_lft forever
5: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether 00:90:27:e7:17:04 brd ff:ff:ff:ff:ff:ff
inet 10.10.10.8/24 brd 10.10.10.255 scope global eth3
valid_lft forever preferred_lft forever
inet 10.10.10.2/24 scope global secondary eth3
valid_lft forever preferred_lft forever
inet6 fe80::290:27ff:fee7:1704/64 scope link
valid_lft forever preferred_lft forever
6: eth4: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master br-lan state DOWN qlen 1000
link/ether 00:90:27:e7:17:05 brd ff:ff:ff:ff:ff:ff
7: eth5: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether 00:90:27:e7:17:06 brd ff:ff:ff:ff:ff:ff
8: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
link/ether 00:90:27:e7:17:01 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.1/24 brd 192.168.1.255 scope global br-lan
valid_lft forever preferred_lft forever
inet6 fda6:e011:5fa9::1/60 scope global noprefixroute
valid_lft forever preferred_lft forever
inet6 fe80::290:27ff:fee7:1701/64 scope link
valid_lft forever preferred_lft forever
14: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN qlen 1000
link/[65534]
inet 192.168.100.1/32 brd 255.255.255.255 scope global wg0
valid_lft forever preferred_lft forever
# cat /etc/config/network
config interface 'loopback'
option device 'lo'
option proto 'static'
option ipaddr '127.0.0.1'
option netmask '255.0.0.0'
config globals 'globals'
option ula_prefix 'fda6:e011:5fa9::/48'
config device
option name 'br-lan'
option type 'bridge'
list ports 'eth0'
list ports 'eth4'
config interface 'lan'
option device 'br-lan'
option proto 'static'
option ipaddr '192.168.1.1'
option netmask '255.255.255.0'
option ip6assign '60'
config interface 'wan'
option device 'eth1'
option proto 'static'
option ipaddr '8.8.8.8'
option netmask '255.255.255.0'
option gateway '8.8.8.1'
option metric '10'
config interface 'wanb'
option proto 'static'
option device 'eth2'
option ipaddr '9.9.9.8'
option netmask '255.255.255.0'
option gateway '9.9.9.1'
option metric '20'
config interface 'lanb'
option proto 'static'
option device 'eth3'
option ipaddr '10.10.10.8'
option netmask '255.255.255.0'
option gateway '8.8.8.1'
config interface 'wg0'
option proto 'wireguard'
option private_key 'GGv/4oUSIBdkUpt6231o/QObwtZ4Rpio9Kbh/Mm/c38='
option listen_port '51820'
list addresses '192.168.100.1'
option defaultroute '0'
config wireguard_wg0
option public_key '0Mjp6LkFoUGx4OFECV71MKqyeJOtbgHnTCQZXxOe8Sw='
option route_allowed_ips '1'
option endpoint_port '51820'
list allowed_ips '192.168.100.0/24'
list allowed_ips '192.168.8.0/24'
list allowed_ips '6.6.6.0/24'
option endpoint_host '7.7.7.8'
option persistent_keepalive '25'
config device
option name 'eth3'
option acceptlocal '1'
************************************
# cat /etc/config/firewall
config defaults
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'REJECT'
config zone
option name 'lan'
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'ACCEPT'
list network 'lan'
list network 'lanb'
list network 'wg0'
config zone
option name 'wan'
option output 'ACCEPT'
option mtu_fix '1'
option masq '1'
option input 'REJECT'
option forward 'REJECT'
list network 'wan'
list network 'wanb'
config forwarding
option src 'lan'
option dest 'wan'
config rule
option name 'Allow-DHCP-Renew'
option src 'wan'
option proto 'udp'
option dest_port '68'
option target 'ACCEPT'
option family 'ipv4'
config rule
option name 'Allow-Ping'
option src 'wan'
option proto 'icmp'
option icmp_type 'echo-request'
option family 'ipv4'
option target 'ACCEPT'
config rule
option name 'Allow-IGMP'
option src 'wan'
option proto 'igmp'
option family 'ipv4'
option target 'ACCEPT'
config rule
option name 'Allow-DHCPv6'
option src 'wan'
option proto 'udp'
option dest_port '546'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-MLD'
option src 'wan'
option proto 'icmp'
option src_ip 'fe80::/10'
list icmp_type '130/0'
list icmp_type '131/0'
list icmp_type '132/0'
list icmp_type '143/0'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-ICMPv6-Input'
option src 'wan'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
list icmp_type 'router-solicitation'
list icmp_type 'neighbour-solicitation'
list icmp_type 'router-advertisement'
list icmp_type 'neighbour-advertisement'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-ICMPv6-Forward'
option src 'wan'
option dest '*'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-IPSec-ESP'
option src 'wan'
option dest 'lan'
option proto 'esp'
option target 'ACCEPT'
config rule
option name 'Allow-ISAKMP'
option src 'wan'
option dest 'lan'
option dest_port '500'
option proto 'udp'
option target 'ACCEPT'
config rule
option name 'wg0'
list proto 'udp'
option src 'wan'
option dest_port '51820'
option target 'ACCEPT'
At the moment, both PCs on either side can ping each other successfully (192.168.8.20 --- 192.168.1.20). However, I am still not completely certain if this is correct. The firewall configurations is same as yesterday
I've been trying to capture Wireguard packets on both OpenWrt devices, but I haven't seen any Wireguard packets at all, which is strange.
I used the command 'tcpdump -w /etc/keepalived/21.cap' to capture packets. However, whenever I run the 'ifdown wg0' command, the ping between 192.168.8.20 and 192.168.1.20 is interrupted. The switch only has routing between the WANs. There's no routing between the LANs (6.6.6.0/24, 10.10.10.0/24, 192.168.1.0/24, and 192.168.8.0/24) on both sides, and LAN traffic is supposed to go through the wg0 interface.
