IPv6 connectivity silently fails after ISP system outage, but IPv4 restores just fine

This has happened twice, so now I'm interested in how to gather any information to see if the problem is OpenWrt or the ISP.

ISP (Charter/Spectrum, Maine, USA), DHCPv6-PD, delegates me a /56, and assigns me a /128 for the router.
Things work fine, I can connect outbound from the LAN or VPN (each has a /64 sliced out of the PD/56). I can connect outbound on IPv4 fine too. Sites like ipquail.com show the IPv4 and IPv6 source addresses, all looks good.

Then something burps in the ISP. Yesterday I got a text message with a service interruption alert. Then later on a service restoration alert.
After service was restored, IPv4 was working fine again. But IPv6 outbound connections timed out. Eventually I realized the IPv6 traffic was not working, so I restarted the wan6 interface ifup wan6. The interface got the same prefix delegated, got the same /128 assigned, and IPv6 connectivity restored.

I theorize that during the ISP outage, something did not get restarted correctly to know to route the IPv6 address and prefix to my OpenWrt router.
But, I think I was still seeing router advertisements/etc. on the WAN interface. (I'll pay more attention this time.)

Next time this happens, what should I look for to understand what's going on? tcpdump for ipv6 traffic on wan6, I presume. I can turn on logging of RAs through an /etc/odhcp6c.user script. Anything else?

I am estimating from ifstatus wan6 that the address assignment is valid for 7 days.
Would this problem have resolved itself at the point when odhcp6c revalidates the assigned address and delegated prefix? (If that's half the valid period, then 3.5 days...yikes!)

Maybe I need a connectivity watchdog that restarts DHCPv6 if IPv4 is working but IPv6 is not?
Is there something the ISP might have forgotten to do to tell all the DHCPv6 clients to restart?

ifstatus output (note the uptime plus the valid time sums up to just shy of 7 days):

{
	"up": true,
	"pending": false,
	"available": true,
	"autostart": true,
	"dynamic": false,
	"uptime": 61038,
	"l3_device": "eth0",
	"proto": "dhcpv6",
	"device": "eth0",
	"metric": 0,
	"dns_metric": 0,
	"delegation": true,
	"ipv4-address": [
		
	],
	"ipv6-address": [
		{
			"address": "2604:<redacted>",
			"mask": 128,
			"preferred": 543757,
			"valid": 543757
		}
	],
	"ipv6-prefix": [
		{
			"address": "2603:<redacted>::",
			"mask": 56,
			"preferred": 543757,
			"valid": 543757,
			"class": "wan6",
			"assigned": {
				"test": {
					"address": "2603:<redacted>::",
					"mask": 64
				},
				"lan": {
					"address": "2603:<redacted>::",
					"mask": 64
				},
				"only6": {
					"address": "2603:<redacted>::",
					"mask": 64
				}
			}
		}
	],
	"ipv6-prefix-assignment": [
		
	],
	"route": [
		{
			"target": "::",
			"mask": 0,
			"nexthop": "fe80::<redacted>",
			"table": 100,
			"source": "::/0"
		},
		{
			"target": "::",
			"mask": 0,
			"nexthop": "fe80::<redacted>",
			"metric": 512,
			"valid": 1798,
			"source": "2603:<redacted>::/56"
		},
		{
			"target": "::",
			"mask": 0,
			"nexthop": "fe80::<redacted>",
			"metric": 512,
			"valid": 1798,
			"source": "2604:<redacted>/128"
		}
	],
	"dns-server": [
		"2001:1998:f00:2::1",
		"2001:1998:f00:1::1"
	],
	"dns-search": [
		
	],
	"neighbors": [
		
	],
	"inactive": {
		"ipv4-address": [
			
		],
		"ipv6-address": [
			
		],
		"route": [
			
		],
		"dns-server": [
			
		],
		"dns-search": [
			
		],
		"neighbors": [
			
		]
	},
	"data": {
		"passthru": "REDACTED"
	}
}

Morning!

From here:

We landed here:

Thank you. My symptoms are not quite the same, but I know what to go look at. I'll monitor what the ISP sends for T1/T2/valid/preferred timings, and see how those compare to the behavior I saw. If necessary, I can use a cron job to make odhcp6c renew earlier than it does today.

Thank you for your post and the insights it provided me. Specifically ipquail and ifstatus wan6; ifstatus wan

Have a good morning and please let other's assist you with the same amount of enthusiasm.

I forced a renew with kill -USR1 $(pidof odhcp6c) while watching with tcpdump. The ISP server is sending this back:

dhcp6 reply (xid=cd95f1
	     (client-ID hwaddr type 1 b039568c224b)
	     (server-ID hwaddr/time type 1 time 494686384 0050569d1430)
	     (IA_NA IAID:1 T1:302400 T2:483840
		    (IA_ADDR 2604:<redacted> pltime:604800 vltime:604800))
	     (IA_PD IAID:1 T1:302400 T2:483840
		    (IA_PD-prefix 2603:<redacted>::/56 pltime:604800 vltime:604800))
	     (DNS-server 2001:1998:f00:2::1 2001:1998:f00:1::1))

With T1 and T2 at 3.5 days and ~5.5 days, my odhcp6c is not misbehaving. It got an address and a delegation less than a day ago, then the ISP had a service outage, and after the restored service the ISP seemingly isn't aware of the previous address/delegation until I restarted the interface.
So I'll call the solution in this case to do a cron job to send USR1 to the odhcp6c daemon to renew regularly to reduce the outage window from 3.5 days to something shorter. And/or to get a fancier probe that checks for IPv6 connectivity periodically and send USR1 if it finds trouble. Hmm, let me look at watchcat!

1 Like

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.