PPPoE IPv6 traffic not being routed

[jamesmacwhite]

I've recently got a FTTP connection with a UK provider (Aquiss) that provides static IPv4 and IPv6 (/56). They use PPPoE with DHCPv6. I am multihomed so this is an additional connection so it named as "wanb"

I have configured my PPPoE WAN interface which successfully connects and provides the static IPv4 and a link-local IPv6. I have set "Obtain IPv6 address" to manual as I want to have full control over the IPv6 alias interface for naming and mwan3 so I don't want it to spawn the virtual interface automatically.

When doing a ping/traceroute, I am finding the traffic is not flowing.

root@linksys-wrt3200acm:~# traceroute -6 -i pppoe-wanb -w 1 ipv6.google.com
traceroute to ipv6.google.com (2a00:1450:4009:823::200e), 30 hops max, 64 byte packets
 1  *  *  *
 2  *  *  *
 3  *  *  *
 4  *  *  *
 5^C

In mwan3 I configured a policy routing rule that when I send traffic to Aquiss IPv6 DNS servers it should use the pppoe-wanb, this works as below. The traffic is going through Aquiss at this point.

root@linksys-wrt3200acm:~# traceroute -6 -i pppoe-wanb -w 1 2001:4d48:face:1::b
traceroute to 2001:4d48:face:1::b (2001:4d48:face:1::b), 30 hops max, 64 byte packets
 1  2001:4d48:feed:97::138 (2001:4d48:feed:97::138)  8.893 ms  9.248 ms  9.002 ms
 2  2001:4d48:feed:97::1 (2001:4d48:feed:97::1)  8.100 ms  8.733 ms  9.078 ms
 3  2001:4d48:feed:99::a (2001:4d48:feed:99::a)  8.997 ms  8.111 ms  7.520 ms
 4  2a05:8940::129 (2a05:8940::129)  13.013 ms  13.072 ms  14.516 ms
 5  *  *  *
 6  *  *  *
 7  *  *  *
 8  wolv-te.core.enta.net (2001:4d48:ace::46)  13.358 ms  13.006 ms  14.304 ms
 9  2001:4d48:feed:9f::b (2001:4d48:feed:9f::b)  14.808 ms  13.874 ms  14.491 ms
10  2001:4d48:face:1::b (2001:4d48:face:1::b)  13.687 ms  13.958 ms  12.238 ms

So it looks like there is IPv6 connectivity, it just doesn't seem to be configured entirely or something is missing.

I'm potentially missing a route somewhere by the looks of it. I have also tried the automatic setting which creates the wanb_6 device, but this also doesn't seem to work unless traffic is specifically told via policy routing to go through.

The PPPoE WAN device is configured like this:

config interface 'wanb'
        option proto 'pppoe'
        option device 'lan2'
        option keepalive '5 5'
        option peerdns '0'
        option metric '20'
        option username 'xxxxxxxxxxxxxx'
        option password xxxxxxxxxxxxxxx'
        option ipv6 '1'

config device
        option name 'pppoe-wanb'

config interface 'wanb6'
        option proto 'dhcpv6'
        option device '@wanb'
        option reqaddress 'try'
        option reqprefix 'auto'
        option peerdns '0'
        option metric '20'

The PPPoE interface has the static IPv4 and a link local IPv6 address fe80::8a9:1d30:c0ec:2122/128, the alias interface for IPv6 as the IPV6-PD showing which has been delegated to the LAN accordingly.

This is what the IPv6 routing table looks like 2001:4d48:ad5d:xxxx::/56 is the prefix from Aquiss, last part blanked:

2001:4d48:ad5d:xxxx::/64 dev br-lan proto static metric 1024 pref medium
unreachable 2001:4d48:ad5d:xxxx::/56 dev lo proto static metric 2147483647 pref medium
2a02:8800:f000:18b0::/64 dev wan proto kernel metric 256 expires 2591995sec pref medium
2a02:88fd:18:a::/64 dev wan proto kernel metric 256 pref medium
fd77:550d:5fb8::/64 dev br-lan proto static metric 1024 pref medium
fd77:550d:5fb8:10::/64 dev wlan1-1 proto static metric 1024 pref medium
unreachable fd77:550d:5fb8::/48 dev lo proto static metric 2147483647 pref medium
fe80::8a9:1d30:c0ec:2122 dev pppoe-wanb proto kernel metric 256 pref medium
fe80::f60f:1bff:fe24:3f00 dev pppoe-wanb metric 1 pref medium
fe80::/64 dev eth0 proto kernel metric 256 pref medium
fe80::/64 dev br-lan proto kernel metric 256 pref medium
fe80::/64 dev wlan0 proto kernel metric 256 pref medium
fe80::/64 dev wan proto kernel metric 256 pref medium
fe80::/64 dev wlan1 proto kernel metric 256 pref medium
fe80::/64 dev wlan1-1 proto kernel metric 256 pref medium
fe80::/64 dev lan2 proto kernel metric 256 pref medium
default dev pppoe-wanb proto static metric 2 pref medium
default via fe80::8a9e:33ff:fef6:7954 dev lan1 proto static metric 3 pref medium
default via fe80::201:5cff:fe9c:2847 dev wan proto ra metric 1024 expires 8995sec pref medium

Any ideas what's missing, I suspect a static route maybe.

[jow]

Usually there's ip rules staged (check ip -6 rule) which should select the interface/route to use by the source address of the outgoing packet.

It might be that the busybox traceroute applet is not smart enough to also select a proper source IP when just given the -i iface option. Before debugging any further, check if traceroute -6 -i pppoe-wanb -s 2001:4d48:ad5d:xxxx::1 -w 1 ipv6.google.com works.

[jamesmacwhite]

@jow Thanks for your reply. That's a good point about busybox limitations with ping and traceroute, however in this case adding the source address explicitly does not work either, so it suggests there is a route missing somewhere. Traffic does flow if I create a mwan3 rule that force the wanb policy, otherwise it appears IPv6 is broken.

It doesn't seem to work with the automatic wanb_6 interface either.

Output of ifstatus wanb6

root@linksys-wrt3200acm:~# ifstatus wanb6
{
        "up": true,
        "pending": false,
        "available": true,
        "autostart": true,
        "dynamic": false,
        "uptime": 40610,
        "l3_device": "pppoe-wanb",
        "proto": "dhcpv6",
        "device": "pppoe-wanb",
        "updated": [
                "routes",
                "prefixes",
                "data"
        ],
        "metric": 20,
        "dns_metric": 0,
        "delegation": true,
        "ipv4-address": [

        ],
        "ipv6-address": [

        ],
        "ipv6-prefix": [
                {
                        "address": "2001:4d48:ad5d:xxxx::",
                        "mask": 56,
                        "preferred": 564190,
                        "valid": 2551390,
                        "class": "wanb6",
                        "assigned": {
                                "lan": {
                                        "address": "2001:4d48:ad5d:xxxx::",
                                        "mask": 60
                                },
                                "guest": {
                                        "address": "2001:4d48:ad5d:xxxx::",
                                        "mask": 64
                                }
                        }
                }
        ],
        "ipv6-prefix-assignment": [

        ],
        "route": [
                {
                        "target": "::",
                        "mask": 0,
                        "nexthop": "::",
                        "metric": 2,
                        "source": "::/0"
                }
        ],
        "dns-server": [

        ],
        "dns-search": [

        ],
        "neighbors": [

        ],
        "inactive": {
                "ipv4-address": [

                ],
                "ipv6-address": [

                ],
                "route": [

                ],
                "dns-server": [
                        "2001:4d48:face:1::b",
                        "2001:4d48:face:4::b"
                ],
                "dns-search": [

                ],
                "neighbors": [

                ]
        },
        "data": {
                "passthru": "0017002020014d48face0001000000000000000b20014d48face0004000000000000000b001f001020014d48012300000000000000000001"
        }
}

[jamesmacwhite]

I'm also suspicious by mwan3, as the interface is being classified as up, as it seems to be able to ping some of the tracking IPs but not all

ping -I 'pppoe-wanb' -c 5 -W 1 '2001:4860:4860::8888' 2>&1

Result:
PING 2001:4860:4860::8888 (2001:4860:4860::8888): 56 data bytes
64 bytes from 2001:4860:4860::8888: seq=0 ttl=118 time=9.007 ms
64 bytes from 2001:4860:4860::8888: seq=1 ttl=118 time=9.033 ms
64 bytes from 2001:4860:4860::8888: seq=2 ttl=118 time=9.378 ms
64 bytes from 2001:4860:4860::8888: seq=3 ttl=118 time=9.399 ms
64 bytes from 2001:4860:4860::8888: seq=4 ttl=118 time=9.646 ms

--- 2001:4860:4860::8888 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 9.007/9.292/9.646 ms
Command:
ping -I 'pppoe-wanb' -c 5 -W 1 '2001:4860:4860::8844' 2>&1

Result:
PING 2001:4860:4860::8844 (2001:4860:4860::8844): 56 data bytes
64 bytes from 2001:4860:4860::8844: seq=0 ttl=118 time=7.847 ms
64 bytes from 2001:4860:4860::8844: seq=1 ttl=118 time=7.971 ms
64 bytes from 2001:4860:4860::8844: seq=2 ttl=118 time=8.249 ms
64 bytes from 2001:4860:4860::8844: seq=3 ttl=118 time=8.414 ms
64 bytes from 2001:4860:4860::8844: seq=4 ttl=118 time=8.486 ms

--- 2001:4860:4860::8844 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 7.847/8.193/8.486 ms
Command:
ping -I 'pppoe-wanb' -c 5 -W 1 '2620:0:ccc::2' 2>&1

Result:
PING 2620:0:ccc::2 (2620:0:ccc::2): 56 data bytes

--- 2620:0:ccc::2 ping statistics ---
5 packets transmitted, 0 packets received, 100% packet loss
Command:
ping -I 'pppoe-wanb' -c 5 -W 1 '2620:0:ccd::2' 2>&1

Result:
PING 2620:0:ccd::2 (2620:0:ccd::2): 56 data bytes

--- 2620:0:ccd::2 ping statistics ---
5 packets transmitted, 0 packets received, 100% packet loss

Not sure how it can ping Google IPv6 DNS servers successfully but fail on OpenDNS ones. It hasn't been marked as down because the reliability settings has met the minimum tracking IPs returning a success.

[jamesmacwhite]

I've found this to be related to metric value issues. It appears the metric field when the interface is DHCPv6, does not respect the value set in LuCI for the IPv6 route.

Other IPv6 routes were taking priority, which was causing the issue.