IPv6 Disappearing after a while for LAN Clients - Troubleshooting impossible due to lack of OpenWRT Logs

I'm trying to get this setup working with IPv6, but so far it's NOT playing nice at all:

  • OpenWRT Router upstream connected to Fiber Convert Box
  • 2 x OPNSense Routers downstream in CARP (High-Availability) Configuration
  • LAN Clients connected to OPNSense Routers (for DHCP, DNS, NTP, ...)

OpenWRT Router is: Linksys WRT1900ACS.

Version Information from the bottom of the OpenWRT Luci Web Page:

Powered by LuCI openwrt-23.05 branch (git-24.086.45142-09d5a38) / OpenWrt 23.05.0 (r23497-6637af95aa)

I also posted on the OPNSense Forum as I'm not really sure what's going on ...

  • OpenWRT Fiber Router is able to ping IPv6 (testing with Google DNS Server 2001:4860:4860::8888) all the time.
  • OPNSense Router1 is able to ping IPv6 (testing with Google DNS Server 2001:4860:4860::8888) all the time.
  • OPNSense Router2 is able to ping IPv6 (testing with Google DNS Server 2001:4860:4860::8888) all the time.

On the other Hand, OPNsense Router DHCP Clients, lose IPv6 Ping "capability" after a while.

Nothing shows up in OPNSense Firewall -> Log Files -> Live View (well, what shows is "Green" :grinning:.

WAN IPv6 (OpenWRT, OPNSense Router1, OPNSense Router2): XXXX:XXXX:XXXX:000**0**:0000:0000:0000:0000/64
LAN IPv6 (OPNSense Router1, OPNSense Router2, LAN Clients): XXXX:XXXX:XXXX:000**1**:0000:0000:0000:0000/64

OpenWRT must have a "Static IPv6 Route" "Back" to the OPNSense CARP IPv6 Address, otherwise it will never work (since OpenWRT only knows about the WAN /64 IPv6 Address and the Subnet it shares with OPNSense i.e. XXXX:XXXX:XXXX:000**0**:0000:0000:0000:0000/64)

So the Static Route is set up as:

  • Target: XXXX:XXXX:XXXX:000**1**:0000:0000:0000:0000/64 (LAN Subnet)
  • Gateway: XXXX:XXXX:XXXX:000**0**:0000:0000:0000:0002 (OPNSense CARP IP Address on the WAN Interface which OpenWRT can "Address")

Advanced Settings have NOT been filled in, not sure if there is something needed here:

However, after a while (can be a few Hours, but definitively it does NOT work after a Day or two), IPv6 on end-clients stops working, while Ping to Google DNS Servers from OpenWRT / OPNSense Router1 / OPNSense Router2 keeps working normally :neutral_face:.

LAN Clients can still ping the OPNSense Router1 (XXXX:XXXX:XXXX:0001::0007), OPNSense Router2 (XXXX:XXXX:XXXX:0001::0008) and OPNSense CARP LAN IPv6 Address (XXXX:XXXX:XXXX:0001::0001) without any Issue.

I'm suspecting an issue where the OPNsense CARP is misconfigured or misbehaving, like Packets are maybe going from LAN Client to OPNSense Router1, go to OpenWRT, then to the Internet, the reply comes back to OpenWRT, forwards to OPNSense CARP Adress and for some reason it goes to OPNSense Router2.

I'm not sure what to think otherwise :woozy_face:.

Unfortunately, OpenWRT makes it nearly impossible to Debug this.
The System and Kernel Logs only show Firewall acting on IPv4 Addresses.
There is ABSOLUTELY nothing in the Logs showing IPv6 Firewall acting on something.

Each Firewall Zone (just the 2 x Default ones) has "Firewall - Zone Settings" checked.

But still, for some Reason, IPv6 is NOT shown in any Firewall, System or Kernel Log.

Any idea what is going on here ?

And how can I enable IPv6 logging on OpenWRT ?

Thank you for your help :smiley:

EDIT 1: once this Issue Occurs, restarting the WAN and WAN6 Interface on OpenWRT seems to solve the issue. Nothing else seemed to help (restarting LAN Client, Restarting OPNSense Router1/Router2).

Once OpenWRT is restarted, the OPNSense LAN Clients can again ping 2001:4860:4860::8888 (Google DNS Servers).

I wonder why this happens though ... IPv6 Connectivity is definitively working for OPNSense Router1 & OPNSense Router2 (I am monitoring 2001:4860:4860::8888 in order to detect WAN Connectivity Link Problems, and the WAN IPv6 Ping works all the time on OPNSense).

So it's not really that the WAN/WAN6 IPv6 Link is Down. It may be more like a Firewall States / Tables / Logs / etc Overload which only solves through an Interface Reboot.

But any idea on how to Investigate Further and get some more Logs for it ?

Use tcpdump via SSH on the openwrt device?

tcpdump -v ip6

to catch all IPv6 packets. You might need to use "-i INTERFACE" to capture on a particular interface.

You could also check into iptables/nftables tracing. That can be really useful when debugging firewall rules.