Ipv6 neighbor/RA not working on LAN

dlakelan · February 13, 2025, 9:14pm

I've got a gl-inet AR750 that I use as a travel router. It isn't getting a global IPv6 on the WAN. However, It's got a ULA set up. However my LAN devices are not getting the ULA. Nothing shows on phone for ULA address...

On my laptop running Linux the ULA doesn't work, ipv6 neighbor discovery doesn't seem to work, nothing works except link local addresses (for example ping ff02::1%wlo1 works).

The OpenWrt device IS sending out RA packets.

Here's some tcp dumps:

From my laptop (shiny)

dlakelan@shiny:~$ sudo tcpdump -c 10 -i wlo1 icmp6
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on wlo1, link-type EN10MB (Ethernet), snapshot length 262144 bytes
13:07:20.394220 IP6 fe80::9683:c4ff:fe06:cf5c > ip6-allnodes: ICMP6, router advertisement, length 120
13:07:46.911697 IP6 shiny > ip6-allnodes: ICMP6, echo request, id 2, seq 1, length 64
13:07:47.223005 IP6 fe80::9683:c4ff:fe06:cf5c > ff02::1:ff67:eab7: ICMP6, neighbor solicitation, who has shiny, length 32
13:07:47.427325 IP6 fe80::828a:7de8:d146:514d > ff02::1:ff67:eab7: ICMP6, neighbor solicitation, who has shiny, length 32
13:07:47.427389 IP6 shiny > fe80::828a:7de8:d146:514d: ICMP6, neighbor advertisement, tgt is shiny, length 32
13:07:47.633604 IP6 fe80::828a:7de8:d146:514d > shiny: ICMP6, echo reply, id 2, seq 1, length 64
13:07:47.911332 IP6 shiny > ip6-allnodes: ICMP6, echo request, id 2, seq 2, length 64
13:07:48.246538 IP6 fe80::9683:c4ff:fe06:cf5c > ff02::1:ff67:eab7: ICMP6, neighbor solicitation, who has shiny, length 32
13:07:48.248551 IP6 fe80::828a:7de8:d146:514d > shiny: ICMP6, echo reply, id 2, seq 2, length 64
13:07:48.911876 IP6 shiny > ip6-allnodes: ICMP6, echo request, id 2, seq 3, length 64
10 packets captured
10 packets received by filter
0 packets dropped by kernel

and from the travel router (dantravel):

root@dantravel:~# tcpdump -c 10 -i br-lan icmp6
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on br-lan, link-type EN10MB (Ethernet), snapshot length 262144 bytes
13:07:20.014191 IP6 fe80::9683:c4ff:fe06:cf5c > ip6-allnodes: ICMP6, router advertisement, length 120
13:07:47.053168 IP6 fe80::2b66:3a00:8967:eab7 > ip6-allnodes: ICMP6, echo request, id 2, seq 1, length 64
13:07:47.053735 IP6 fe80::9683:c4ff:fe06:cf5c > ff02::1:ff67:eab7: ICMP6, neighbor solicitation, who has fe80::2b66:3a00:8967:eab7, length 32
13:07:47.313346 IP6 fe80::828a:7de8:d146:514d > ff02::1:ff67:eab7: ICMP6, neighbor solicitation, who has fe80::2b66:3a00:8967:eab7, length 32
13:07:47.467867 IP6 fe80::2b66:3a00:8967:eab7 > fe80::828a:7de8:d146:514d: ICMP6, neighbor advertisement, tgt is fe80::2b66:3a00:8967:eab7, length 32
13:07:47.582811 IP6 fe80::828a:7de8:d146:514d > fe80::2b66:3a00:8967:eab7: ICMP6, echo reply, id 2, seq 1, length 64
13:07:48.077163 IP6 fe80::2b66:3a00:8967:eab7 > ip6-allnodes: ICMP6, echo request, id 2, seq 2, length 64
13:07:48.120642 IP6 fe80::9683:c4ff:fe06:cf5c > ff02::1:ff67:eab7: ICMP6, neighbor solicitation, who has fe80::2b66:3a00:8967:eab7, length 32
13:07:48.142663 IP6 fe80::828a:7de8:d146:514d > fe80::2b66:3a00:8967:eab7: ICMP6, echo reply, id 2, seq 2, length 64
13:07:49.039306 IP6 fe80::2b66:3a00:8967:eab7 > ip6-allnodes: ICMP6, echo request, id 2, seq 3, length 64
10 packets captured
10 packets received by filter
0 packets dropped by kernel

This was while doing ping ff02::1%wlo1 from laptop.

After this, still no ULA addresses on shiny, despite you can see the RA was sent and received.

Note that this laptop works fine when I'm at home and have a TP-link omada access point going. So it's not like ipv6 doesn't work at all on my laptop.

The laptop is running Debian testing, with NetworkManager. The router is running OpenWrt 23.05.2 r23630-842932a63d

Any ideas what's up here?

dlakelan · February 13, 2025, 9:28pm

Further wiresharking shows that my laptop is sending router solicitations and the router is NOT responding with router adverts.

_bernd · February 13, 2025, 11:58pm

What's the content of etc config dhcp? ....

dlakelan · February 14, 2025, 12:23am

root@dantravel:~# cat /etc/config/dhcp 

config dnsmasq
	option domainneeded '1'
	option localise_queries '1'
	option rebind_protection '1'
	option rebind_localhost '1'
	option local '/lan/'
	option domain 'lan'
	option expandhosts '1'
	option authoritative '1'
	option readethers '1'
	option leasefile '/tmp/dhcp.leases'
	option resolvfile '/tmp/resolv.conf.d/resolv.conf.auto'
	option localservice '1'
	option cachesize '2000'
	list server '1.1.1.1'
	list server '1.0.0.1'
	list server '2606:4700:4700::1111'
	list server '2606:4700:4700::1001'

config dhcp 'lan'
	option interface 'lan'
	option start '100'
	option limit '150'
	option leasetime '12h'
	option ra 'server'
	option preferred_lifetime '3h'
	option dhcpv6 'server'
	option ra_default '2'

config dhcp 'wan'
	option interface 'wan'
	option ignore '1'

config odhcpd 'odhcpd'
	option maindhcp '0'
	option leasefile '/tmp/hosts/odhcpd'
	option leasetrigger '/usr/sbin/odhcpd-update'
	option loglevel '4'

config dhcp 'wan6'
	option interface 'wan6'
	option ignore '1'
	option master '1'

And then a bunch of static hostnames which I elided

I also tried some other things, like hybrid mode for lan, but none of them worked either.

dave14305 · February 14, 2025, 12:43am

Is there anything abnormal in the RAs? Are the lifetimes gt 0?

tcpdump -p -v -n -e -i br-lan '(icmp6 and ip6[40] == 134) or (udp and port 547)'

I guess you can compare the RA received on the laptop to make sure no other device is sending RAs.

dlakelan · February 14, 2025, 2:46am

looking at the laptop on wireshark, when I restart odhcpd it sends a router advertisement, the advertisement has 1800s lifetime, it advertises the correct ULA prefix, it includes the correct DNS address, and such... But the laptop doesn't add the prefix to the interface, and neither does my phone or tablets.

there's the wireguard info on the received packet, coming from the router.

klipz · February 14, 2025, 3:53am

Following, because I have a very similar (identical?) problem with a host on my network and OpenWRT 24.10 router upstream on the LAN.

From host (Alpine, running dhcpcd):

Feb 13 22:02:47 daemon.info dhcpcd[5633]: eth3: soliciting an IPv6 router

12 seconds later...

Feb 13 22:02:59 daemon.warn dhcpcd[5633]: eth3: no IPv6 Routers available

One and a half long minutes after this...

Feb 13 22:04:29 daemon.info dhcpcd[5633]: eth3: Router Advertisement from fe80::......
Feb 13 22:04:29 daemon.info dhcpcd[5633]: eth3: adding address 2600:......
Feb 13 22:04:29 daemon.info dhcpcd[5633]: eth3: adding address fdfa:......
Feb 13 22:04:29 daemon.info dhcpcd[5633]: eth3: adding route to 2600:......

As shown, after my host sees the unsolicited RA from OpenWRT on the LAN, all addresses and routes are added as expected and everything works great.

The big question is, why does OpenWRT's DHCPv6 server not reply to my dhcpcd client's router solicitation?

dlakelan · February 14, 2025, 4:43am

The weird thing is my client doesn't ever get an IPv6, even once an unsolicited RA shows up.

dlakelan · February 14, 2025, 10:46pm

I tried an upgrade to 23.05.5 to see if anything changed, but it didn't. Still no replies to my laptop's router solicitation, still no prefix after the unsolicited RAs

Some more info from wireguard snooping. It appears to me that neighbor discovery isn't working. Could there be some kind of issue with multicast snooping in br-lan breaking neighbor discovery?

dlakelan · February 16, 2025, 1:39am

Here's tcpdump on the router when I have my Android phone join. (note that this phone gets ipv6 just fine at home, or other networks where i've had ipv6)

the link layer address ending in 514d is the phone.
You can see when it joins it sends a router solicitation, the router then sends a router advertisement. The phone responds with yet another router solicitation. the router sends another router advert. The router then sends a neighbor solicitation for the link layer address of the phone. The phone doesn't respond, the router repeatedly sends this NDP solicitation, and the phone never responds.

This cycle repeats... So either the phone is never getting the neighbor solicitation, or the phone is sending neighbor replies and they aren't being received by the router. all of this is over the wifi.

17:29:05.071569 IP6 fe80::828a:7de8:d146:514d > ip6-allrouters: ICMP6, router solicitation, length 16
17:29:05.072604 IP6 fe80::9683:c4ff:fe06:cf5c > fe80::828a:7de8:d146:514d: ICMP6, router advertisement, length 96
17:29:09.680077 IP6 fe80::828a:7de8:d146:514d > ip6-allrouters: ICMP6, router solicitation, length 16
17:29:09.681015 IP6 fe80::9683:c4ff:fe06:cf5c > fe80::828a:7de8:d146:514d: ICMP6, router advertisement, length 96
17:29:10.120861 IP6 fe80::9683:c4ff:fe06:cf5c > fe80::828a:7de8:d146:514d: ICMP6, neighbor solicitation, who has fe80::828a:7de8:d146:514d, length 32
17:29:11.160874 IP6 fe80::9683:c4ff:fe06:cf5c > fe80::828a:7de8:d146:514d: ICMP6, neighbor solicitation, who has fe80::828a:7de8:d146:514d, length 32
17:29:12.200917 IP6 fe80::9683:c4ff:fe06:cf5c > fe80::828a:7de8:d146:514d: ICMP6, neighbor solicitation, who has fe80::828a:7de8:d146:514d, length 32
17:29:18.796044 IP6 fe80::828a:7de8:d146:514d > ip6-allrouters: ICMP6, router solicitation, length 16
17:29:18.796934 IP6 fe80::9683:c4ff:fe06:cf5c > fe80::828a:7de8:d146:514d: ICMP6, router advertisement, length 96
17:29:23.800873 IP6 fe80::9683:c4ff:fe06:cf5c > fe80::828a:7de8:d146:514d: ICMP6, neighbor solicitation, who has fe80::828a:7de8:d146:514d, length 32
17:29:24.840875 IP6 fe80::9683:c4ff:fe06:cf5c > fe80::828a:7de8:d146:514d: ICMP6, neighbor solicitation, who has fe80::828a:7de8:d146:514d, length 32
17:29:25.880900 IP6 fe80::9683:c4ff:fe06:cf5c > fe80::828a:7de8:d146:514d: ICMP6, neighbor solicitation, who has fe80::828a:7de8:d146:514d, length 32
17:29:29.338302 IP6 fe80::c43f:2b77:5a82:ea8c > ip6-allrouters: ICMP6, router solicitation, length 16
17:29:29.339227 IP6 fe80::9683:c4ff:fe06:cf5c > fe80::c43f:2b77:5a82:ea8c: ICMP6, router advertisement, length 96
17:29:34.360865 IP6 fe80::9683:c4ff:fe06:cf5c > fe80::c43f:2b77:5a82:ea8c: ICMP6, neighbor solicitation, who has fe80::c43f:2b77:5a82:ea8c, length 32
17:29:35.400879 IP6 fe80::9683:c4ff:fe06:cf5c > fe80::c43f:2b77:5a82:ea8c: ICMP6, neighbor solicitation, who has fe80::c43f:2b77:5a82:ea8c, length 32
17:29:36.440891 IP6 fe80::9683:c4ff:fe06:cf5c > fe80::c43f:2b77:5a82:ea8c: ICMP6, neighbor solicitation, who has fe80::c43f:2b77:5a82:ea8c, length 32
17:29:37.310294 IP6 fe80::828a:7de8:d146:514d > ip6-allrouters: ICMP6, router solicitation, length 16
17:29:37.311248 IP6 fe80::9683:c4ff:fe06:cf5c > fe80::828a:7de8:d146:514d: ICMP6, router advertisement, length 96
17:29:42.360886 IP6 fe80::9683:c4ff:fe06:cf5c > fe80::828a:7de8:d146:514d: ICMP6, neighbor solicitation, who has fe80::828a:7de8:d146:514d, length 32
17:29:43.400854 IP6 fe80::9683:c4ff:fe06:cf5c > fe80::828a:7de8:d146:514d: ICMP6, neighbor solicitation, who has fe80::828a:7de8:d146:514d, length 32
17:29:44.440879 IP6 fe80::9683:c4ff:fe06:cf5c > fe80::828a:7de8:d146:514d: ICMP6, neighbor solicitation, who has fe80::828a:7de8:d146:514d, length 32

Now, here's what happens when my linux laptop (ll address ending in eab7) tries to connect. Here's on the router:

17:34:44.384064 IP6 fe80::2b66:3a00:8967:eab7 > ip6-allrouters: ICMP6, router solicitation, length 8
17:34:44.384953 IP6 fe80::9683:c4ff:fe06:cf5c > ff02::1:ff67:eab7: ICMP6, neighbor solicitation, who has fe80::2b66:3a00:8967:eab7, length 32
17:34:45.400885 IP6 fe80::9683:c4ff:fe06:cf5c > ff02::1:ff67:eab7: ICMP6, neighbor solicitation, who has fe80::2b66:3a00:8967:eab7, length 32
17:34:46.440879 IP6 fe80::9683:c4ff:fe06:cf5c > ff02::1:ff67:eab7: ICMP6, neighbor solicitation, who has fe80::2b66:3a00:8967:eab7, length 32
17:34:48.717477 IP6 fe80::2b66:3a00:8967:eab7 > ip6-allrouters: ICMP6, router solicitation, length 8
17:34:48.719004 IP6 fe80::9683:c4ff:fe06:cf5c > ff02::1:ff67:eab7: ICMP6, neighbor solicitation, who has fe80::2b66:3a00:8967:eab7, length 32
17:34:49.720872 IP6 fe80::9683:c4ff:fe06:cf5c > ff02::1:ff67:eab7: ICMP6, neighbor solicitation, who has fe80::2b66:3a00:8967:eab7, length 32
17:34:50.760892 IP6 fe80::9683:c4ff:fe06:cf5c > ff02::1:ff67:eab7: ICMP6, neighbor solicitation, who has fe80::2b66:3a00:8967:eab7, length 32
17:34:56.980823 IP6 fe80::2b66:3a00:8967:eab7 > ip6-allrouters: ICMP6, router solicitation, length 8
17:34:56.981892 IP6 fe80::9683:c4ff:fe06:cf5c > ff02::1:ff67:eab7: ICMP6, neighbor solicitation, who has fe80::2b66:3a00:8967:eab7, length 32
17:34:58.040864 IP6 fe80::9683:c4ff:fe06:cf5c > ff02::1:ff67:eab7: ICMP6, neighbor solicitation, who has fe80::2b66:3a00:8967:eab7, length 32
17:34:59.080891 IP6 fe80::9683:c4ff:fe06:cf5c > ff02::1:ff67:eab7: ICMP6, neighbor solicitation, who has fe80::2b66:3a00:8967:eab7, length 32

and at the same time, here's on the laptop:

17:34:43.255457 IP6 :: > ff02::1:ff67:eab7: ICMP6, neighbor solicitation, who has fe80::2b66:3a00:8967:eab7, length 32
17:34:44.355237 IP6 fe80::2b66:3a00:8967:eab7 > ip6-allrouters: ICMP6, router solicitation, length 8
17:34:44.478699 IP6 fe80::9683:c4ff:fe06:cf5c > ff02::1:ff67:eab7: ICMP6, neighbor solicitation, who has fe80::2b66:3a00:8967:eab7, length 32
17:34:46.526169 IP6 fe80::9683:c4ff:fe06:cf5c > ff02::1:ff67:eab7: ICMP6, neighbor solicitation, who has fe80::2b66:3a00:8967:eab7, length 32
17:34:48.652529 IP6 fe80::2b66:3a00:8967:eab7 > ip6-allrouters: ICMP6, router solicitation, length 8
17:34:49.802988 IP6 fe80::9683:c4ff:fe06:cf5c > ff02::1:ff67:eab7: ICMP6, neighbor solicitation, who has fe80::2b66:3a00:8967:eab7, length 32
17:34:50.826959 IP6 fe80::9683:c4ff:fe06:cf5c > ff02::1:ff67:eab7: ICMP6, neighbor solicitation, who has fe80::2b66:3a00:8967:eab7, length 32
17:34:56.906275 IP6 fe80::2b66:3a00:8967:eab7 > ip6-allrouters: ICMP6, router solicitation, length 8
17:34:56.971146 IP6 fe80::9683:c4ff:fe06:cf5c > ff02::1:ff67:eab7: ICMP6, neighbor solicitation, who has fe80::2b66:3a00:8967:eab7, length 32
17:34:58.199718 IP6 fe80::9683:c4ff:fe06:cf5c > ff02::1:ff67:eab7: ICMP6, neighbor solicitation, who has fe80::2b66:3a00:8967:eab7, length 32
17:34:59.223756 IP6 fe80::9683:c4ff:fe06:cf5c > ff02::1:ff67:eab7: ICMP6, neighbor solicitation, who has fe80::2b66:3a00:8967:eab7, length 32

You can see the laptop sends a router solicitation, and the router then sends neighbor solicitations, which the laptop doesn't respond to.

All of this is supremely weird. I've never seen anything like this behavior before and have always had ipv6 "just work".

dlakelan · February 21, 2025, 2:03pm

So in the end I bought myself a gl-inet Beryl as an upgrade, installed OpenWrt right away, configured it similarly to the other device, and swapped them out. IPv6 works fine on the beryl so I'm guessing it's some kind of kernel bug that affects the older device. It's not worth it to me to spend time figuring out the issue so the solution was to just get something that works.

dlakelan · February 24, 2025, 10:03pm

Ok, weird crap going on now. The Beryl worked for 3 days, and this morning it's not working anymore. Even after a reboot, wifi clients don't get ipv6 ULA prefix! I'm going to try power cycling it.

dave14305 · February 24, 2025, 10:05pm

Anything magical about the 3 days period? Did any IPv4 lease renew around that time? Stuff like that?

dlakelan · February 24, 2025, 10:07pm

I restarted the WAN this morning because I switched the WAN to my other hotspot. But this is LAN ULA that's missing... I turned it off and on again just now... why does it not distribute ULA anymore? It literally worked for the last few days.

Again, ipv6 neighbor discovery is failing:

root@dantravel:~# ip -6 neigh
fe80::1296:93ff:fea9:ac47 dev br-lan lladdr 10:96:93:a9:ac:47 used 0/0/0 probes 1 STALE
fe80::2b66:3a00:8967:eab7 dev br-lan  ref 1 used 0/0/0 probes 6 INCOMPLETE
fe80::828a:7de8:d146:514d dev br-lan lladdr d8:cf:bf:76:83:9b used 0/0/0 probes 2 STALE
fe80::7658:f3ff:fe78:a7c0 dev br-lan lladdr 74:58:f3:78:a7:c0 used 0/0/0 probes 1 STALE
fe80::1448:d878:b025:8dd6 dev br-lan  used 0/0/0 probes 6 FAILED
fe80::fe49:2dff:fe2e:eae9 dev br-lan lladdr fc:49:2d:2e:ea:e9 used 0/0/0 probes 1 STALE

This is after pinging ff02::1%br-lan from the router, and ff02::1%wlo1 from my laptop. The laptop gets replies from my phone.

dave14305 · February 24, 2025, 10:20pm

Have you tried increasing the log level of odhcpd?

uci set dhcp.odhcpd.loglevel='7'
uci commit

Then restart odhcpd. Maybe it will confess…

dlakelan · February 24, 2025, 10:27pm

It says:

Mon Feb 24 14:25:26 2025 daemon.debug odhcpd[4964]: Using a RA lifetime of 1800 seconds on lan
Mon Feb 24 14:25:26 2025 daemon.info odhcpd[4964]: Address fd7c:3786:281f::1 (dprefix 0, valid 4294967295) not suitable as RA route on lan
Mon Feb 24 14:25:26 2025 daemon.notice odhcpd[4964]: Sending a RA on lan
Mon Feb 24 14:25:26 2025 daemon.debug odhcpd[4964]: Sent 96 bytes to ff02::1%lan@br-lan

That line about "not suitable as RA" is interesting...

dave14305 · February 24, 2025, 10:38pm

What’s the prefix length of the ULA in the network config?

dlakelan · February 24, 2025, 10:40pm

it's /48 like default for OpenWrt

dlakelan · February 24, 2025, 10:51pm

that's what's received on the laptop. Router lifetime is 1800 seconds. Everything looks normal to me. But note that the laptop can't neighbor discover the router:

here's from teh laptop looking for the router's LL address

ip -6 neigh 
...
fe80::9683:c4ff:fe5f:b549 dev wlo1 FAILED

One thing that might be different is I think I was getting a public address before on the other hotspot, but now am not showing a public v6 address on WAN with this hotspot.

dave14305 · February 24, 2025, 10:55pm

Is there a lifetime shown under the Prefix option in Wireshark?