Helping with Wireguard IPV6 firewall traffic rule--or am I being QOSed by ISP?

Hi there,

I have been using wireguard on a windows 10 PC to access a remote openwrt 21.02 snapshot router successfully with ipv4 for a while. Recently I used ipv6 address for the connection, and experienced frequent interruption. The phenomenon is a connected WG route suddenly "freeze" on all remote pages and ssh content, but if I do a reconnection then remote content can be displayed and updated again, a while later the "freeze" reappear.

Remote peer has public/native ipv4/v6 addresses, no NAT involved. My win10 PC dwells behind another openwrt router, which also has both native ipv4/v6 addresses but with different ISP, but I think the client side is irrelevant, no?

The firewall rule on the remote openwrt router is as below, WG listens on 65000 while 192.168.88.1 is remote router lan ip:

config defaults
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option synflood_protect '1'
	option flow_offloading '1'

config zone
	option name 'lan'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'
	list network 'lan'
	list network 'WireGuard'

config zone
	option name 'wan'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'
	list network 'wan'
	list network 'wan6'

config forwarding
	option src 'lan'
	option dest 'wan'

config rule
	option name 'WireGuard'
	option src 'wan'
	option dest_port '65000'
	option target 'ACCEPT'
	list proto 'udp'

config rule
	option name 'Allow-DHCP-Renew'
	option src 'wan'
	option proto 'udp'
	option dest_port '68'
	option target 'ACCEPT'
	option family 'ipv4'

config rule
	option name 'Allow-Ping'
	option src 'wan'
	option proto 'icmp'
	option icmp_type 'echo-request'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-IGMP'
	option src 'wan'
	option proto 'igmp'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-DHCPv6'
	option src 'wan'
	option proto 'udp'
	option src_ip 'fc00::/6'
	option dest_ip 'fc00::/6'
	option dest_port '546'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-MLD'
	option src 'wan'
	option proto 'icmp'
	option src_ip 'fe80::/10'
	list icmp_type '130/0'
	list icmp_type '131/0'
	list icmp_type '132/0'
	list icmp_type '143/0'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Input'
	option src 'wan'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	list icmp_type 'router-solicitation'
	list icmp_type 'neighbour-solicitation'
	list icmp_type 'router-advertisement'
	list icmp_type 'neighbour-advertisement'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Forward'
	option src 'wan'
	option proto 'icmp'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'
	list icmp_type 'bad-header'
	list icmp_type 'destination-unreachable'
	list icmp_type 'echo-reply'
	list icmp_type 'echo-request'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'unknown-header-type'
	option dest '*'

config rule
	option name 'Allow-IPSec-ESP'
	option src 'wan'
	option proto 'esp'
	option target 'ACCEPT'
	option dest 'lan'

config rule
	option name 'Allow-ISAKMP'
	option src 'wan'
	option dest_port '500'
	option proto 'udp'
	option target 'ACCEPT'
	option dest 'lan'

config rule
	option name 'Support-UDP-Traceroute'
	option src 'wan'
	option dest_port '33434:33689'
	option proto 'udp'
	option family 'ipv4'
	option target 'REJECT'
	option enabled '0'

config include
	option path '/etc/firewall.user'

config redirect
	option target 'DNAT'
	option name 'WireGuard'
	list proto 'udp'
	option src 'wan'
	option src_dport '65000'
	option dest 'lan'
	option dest_port '65000'
	option dest_ip '192.168.88.1'

config include 'miniupnpd'
	option type 'script'
	option path '/usr/share/miniupnpd/firewall.include'
	option family 'any'
	option reload '1'

If I use ipv4 address on remote access then this random "freeze/interruption" will gone. So can anyone tell me if I'm creating correct firewall rules? Or am I being QOSed by ISP specifically on IPV6 UDP? How can I have more log/detail about this freeze/interruption on the client's side? Thx in advance.

This redirect is not necessary, you have a rule to allow and that is enough.

One low level way is to monitor the packets and see if there are any mistakes or just silence from the other side.
opkg update; opkg install tcpdump; tcpdump -i WireGuard -n

Thanks, I deleted the NAT rule and confirm accessing is still OK. However the "freeze" still exist, strangely no freeze when using ipv4 address, not at all!
Sadly remote peer router has only an 8 MB ROM and cannot install tcpdump package, is there anything I can do on my client's side?

OK so on windows wg client has the following logs:

2022-04-07 21:19:55.831: [TUN] [WGP6] Handshake for peer 1 ([v6 address]:65000) did not complete after 5 seconds, retrying (try 2
2022-04-07 21:19:55.831: [TUN] [WGP6] Sending handshake initiation to peer 1 ([v6 address]:65000)
2022-04-07 21:20:00.852: [TUN] [WGP6] Handshake for peer 1 ([v6 address]:65000) did not complete after 5 seconds, retrying (try 3
2022-04-07 21:20:00.852: [TUN] [WGP6] Sending handshake initiation to peer 1 ([v6 address]:65000)
2022-04-07 21:20:05.896: [TUN] [WGP6] Handshake for peer 1 ([v6 address]:65000) did not complete after 5 seconds, retrying (try 4
2022-04-07 21:20:05.896: [TUN] [WGP6] Sending handshake initiation to peer 1 ([v6 address]:65000)
2022-04-07 21:20:10.917: [TUN] [WGP6] Handshake for peer 1 ([v6 address]:65000) did not complete after 5 seconds, retrying (try 5

So does this means remote peer not responding?

I was going to say that you can run wireshark on Windows and monitor the traffic in the tunnel.
However if there are handshakes lost it is a bad sign that packets are not getting to the server.
Have you tried to run a continuous ping6 to the server?

tired ping v6 from op router on my side and with a slight 7% packet loss on round 1:

51 packets transmitted, 47 packets received, 7% packet loss
round-trip min/avg/max = 4.192/70.279/267.182 ms

and round 2 9%:

51 packets transmitted, 46 packets received, 9% packet loss
round-trip min/avg/max = 4.227/79.771/280.969 ms

while v4 has 0% loss on round1 and 7% on round 2:

51 packets transmitted, 51 packets received, 0% packet loss
round-trip min/avg/max = 4.555/25.370/88.231 ms
51 packets transmitted, 47 packets received, 7% packet loss
round-trip min/avg/max = 4.041/26.676/73.851 ms

But the max latency for v6 is much higher, what the heck?! Any sign of QOS from ISP?

Try mtr to identify the intermediate hops as well.

1
Nothing special found, 9 hops in both v4 and v6. so strange....

Tried another round of ping and ping6, no packet loss this time, maybe because the night peak?
However the freeze happens all day long even in the afternoon.

From the evidence I don't see anything wrong. It would be nice to be able to troubleshoot from the OpenWrt side as well and verify if you can see packets arriving or not. But from what we have we cannot point the finger somewhere. I guess you could raise an issue with your provider, but there's no guarantee you'll get a concrete answer. My last advice would be to change the WG ports.

Is it related to this bug? I have enabled soft flow-offload on the remote router

IPV6 flow offload broken

I'm using 21.02 snapshot r16543-ee62912b2d, which is still on 21.02 branch.

You can try to turn it off and see how it goes.

I turned off flow-offload on remote router and changed WG port, still the same.
however when I turned off flow-offload on my home op router this freeze is gone, I can have stable WG connection for 3 hours. But my home router is MT7621 based, without flow-offload the network becomes laggy.

So I will treat this as a bug triggered by op flow-offload. For now I will use v4 address when connecting WG peer.

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.