Thank you for the suggestions. I'm trying to make the problem reproducible at the moment. I have a GigE from my ISP which terminates in a 'breakout box' with four GigE interfaces on it, which appear to be switched i.e. Wireshark on one interface doesn't see traffic over another interface..
I have up to three routers connected, each to its own GigE interface:
- a TP-Link Archer C7 v2 running OpenWrt 18.06.8
- a TP-Link Archer C7 v4 running 19.07.4
- an Inteno EX400 running EX400-X-INT-4.1.6-190404_1753 kernel 4.4.14 ( IOPSYS is OpenWrt based)
I do not have root or command line access on the Inteno, as it is supplied by the ISP.
On all three routers, IPv4 is rock solid.
On all three router IPv6 is unstable. Pinging remote IPv6 devices I get packet loss up to 80-odd per-cent, and network timeouts. It plays merry hell with the 'Happy Eyeballs' implementation in Firefox, as IPv6 can appear to be working, then time out for extended periods before returning.
The upstream ISP device has a Huawei MAC address.
I have adequate understanding of IPv4, but I am not expert on the niceties of IPv6 setup and routing.
After an interface restart, it can take 10-20 minutes before the WAN6 interface comes up and gets IPv6 addresses.
I can run Wireshark on a PC, but I don't currently have access to a switch with mirroring capability. I can add mirroring on the TP-Link Archer C7 v4, but I'm not sure of the config to stop it trying to interfere and route when all I want is a switch with LAN1 connected to the upstream WAN, LAN2 as the mirror port of LAN1 and LAN2 connected to the downstream router. I may well shell out for a TP-Link TL-SG108E switch just to make things easy (and I can find a use for it later).
I'm also getting instances of the "ping: sendto: Permission denied" error (documented elsewhere in this forum), so something is not working well.
https://unix.stackexchange.com/questions/581753/what-is-ping6-sendto-permission-denied-a-sign-of-when-debugging
https://forum.openwrt.org/t/ping6-sendto-permission-denied-on-some-ipv6-addresses/61163
Essential, the TP-Link Archer C7 v4 running 19.07.4 is the 'crash test dummy', and available for weird and wonderful configs and can be rebooted without problems. The Inteno is ISP supplied and locked down, but experiences the same IPv6 issues, and the Archer C7 v2 running 18.06.8 is the production router that I'd prefer not to mess with until I have a better working config. It has IPv6 issues too.
I put a notebook PC running Linux directly attached to the ISP breakout and captured interface startup and pinging the problematic destinations from the PC with Wireshark, but could see no obvious problems. It seems to be a specific issue with OpenWrt, or possibly a specific combination of OpenWrt and Huawei: if this were a general problem, more people than me would be tearing their hair out in frustration. I want to get IPv6 working reliably.
I did a power-off->power-on restart of the OpenWrt 19.07.04 router while composing this posting. IPv4 is up. The WAN6 interface is stubbornly still down after seventeen (17) came up after 35 (thirty-five) minutes.
Edit to Add:
Sun Oct 25 21:37:13 2020 kern.notice kernel: [ 0.000000] Linux version 4.14.195 (builder@buildhost) (gcc version 7.5.0 (OpenWrt GCC 7.5.0 r11208-ce6496d796)) #0 Sun Sep 6 16:19:39 2020
...
Sun Oct 25 22:12:57 2020 daemon.notice netifd: Interface 'wan6' is now up
...and while perusing the kernel log, I saw this. Hmm.
Sun Oct 25 22:12:58 2020 daemon.warn odhcpd[1244]: A default route is present but there is no public prefix on lan thus we don't announce a default route!
While interface down:
# ifstatus wan6
{
"up": false,
"pending": true,
"available": true,
"autostart": true,
"dynamic": false,
"proto": "dhcpv6",
"device": "eth0.2",
"data": {
}
}
# ifstatus lan
{
"up": true,
"pending": false,
"available": true,
"autostart": true,
"dynamic": false,
"uptime": 941,
"l3_device": "br-lan",
"proto": "static",
"device": "br-lan",
"updated": [
"addresses"
],
"metric": 0,
"dns_metric": 0,
"delegation": true,
"ipv4-address": [
{
"address": "192.168.8.1",
"mask": 24
}
],
"ipv6-address": [
],
"ipv6-prefix": [
],
"ipv6-prefix-assignment": [
{
"address": "fdb6:1bed:70ff::",
"mask": 60,
"local-address": {
"address": "fdb6:1bed:70ff::1",
"mask": 60
}
}
],
"route": [
],
"dns-server": [
],
"dns-search": [
],
"neighbors": [
],
"inactive": {
"ipv4-address": [
],
"ipv6-address": [
],
"route": [
],
"dns-server": [
],
"dns-search": [
],
"neighbors": [
]
},
"data": {
}
}
# ip -6 address show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1000
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
inet6 fe80::724f:57ff:fe8a:755c/64 scope link
valid_lft forever preferred_lft forever
6: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
inet6 fdb6:1bed:70ff::1/60 scope global
valid_lft forever preferred_lft forever
inet6 fe80::724f:57ff:fe8a:755c/64 scope link
valid_lft forever preferred_lft forever
8: eth0.2@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
inet6 fe80::724f:57ff:fe8a:755d/64 scope link
valid_lft forever preferred_lft forever
9: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
inet6 fe80::724f:57ff:fe8a:755b/64 scope link
valid_lft forever preferred_lft forever
10: wlan1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
inet6 fe80::724f:57ff:fe8a:755c/64 scope link
valid_lft forever preferred_lft forever
# ip -6 route show table all
fdb6:1bed:70ff::/64 dev br-lan metric 1024
unreachable fdb6:1bed:70ff::/48 dev lo metric 2147483647 error -148
fe80::/64 dev eth0 metric 256
fe80::/64 dev eth0.2 metric 256
fe80::/64 dev br-lan metric 256
fe80::/64 dev wlan1 metric 256
fe80::/64 dev wlan0 metric 256
local ::1 dev lo table local metric 0
anycast fdb6:1bed:70ff:: dev br-lan table local metric 0
local fdb6:1bed:70ff::1 dev br-lan table local metric 0
anycast fe80:: dev eth0.2 table local metric 0
anycast fe80:: dev eth0 table local metric 0
anycast fe80:: dev br-lan table local metric 0
anycast fe80:: dev wlan1 table local metric 0
anycast fe80:: dev wlan0 table local metric 0
local fe80::724f:57ff:fe8a:755b dev wlan0 table local metric 0
local fe80::724f:57ff:fe8a:755c dev eth0 table local metric 0
local fe80::724f:57ff:fe8a:755c dev br-lan table local metric 0
local fe80::724f:57ff:fe8a:755c dev wlan1 table local metric 0
local fe80::724f:57ff:fe8a:755d dev eth0.2 table local metric 0
ff00::/8 dev eth0 table local metric 256
ff00::/8 dev br-lan table local metric 256
ff00::/8 dev eth0.2 table local metric 256
ff00::/8 dev wlan1 table local metric 256
ff00::/8 dev wlan0 table local metric 256
# ip -6 neigh show
fe80::30ae:8087:841d:c540 dev br-lan lladdr b8:86:87:5f:42:eb used 0/0/0 probes 4 STALE
fe80::ded2:fcff:fe03:782d dev eth0.2 lladdr dc:d2:fc:03:78:2d router ref 1 used 0/0/0 probes 1 REACHABLE
fe80::724f:57ff:fe8a:755d dev eth0.2 lladdr 70:4f:57:8a:75:5d used 0/0/0 probes 0 STALE</code>
<code># ip -6 rule show
0: from all lookup local
32766: from all lookup main
4200000001: from all iif lo lookup unspec 12
4200000006: from all iif br-lan lookup unspec 12
4200000008: from all iif eth0.2 lookup unspec 12
After interface came up:
# ifstatus wan6
{
"up": true,
"pending": false,
"available": true,
"autostart": true,
"dynamic": false,
"uptime": 488,
"l3_device": "eth0.2",
"proto": "dhcpv6",
"device": "eth0.2",
"updated": [
"addresses",
"routes",
"prefixes",
"data"
],
"metric": 0,
"dns_metric": 0,
"delegation": true,
"ipv4-address": [
],
"ipv6-address": [
{
"address": "2a0a:2780:4fff:1dd8:b514:567b:2a52:30d0",
"mask": 128,
"preferred": 1752,
"valid": 3102
}
],
"ipv6-prefix": [
{
"address": "2a0a:2780:4f4c::",
"mask": 48,
"preferred": 1752,
"valid": 3102,
"class": "wan6",
"assigned": {
"lan": {
"address": "2a0a:2780:4f4c::",
"mask": 60
},
"wireless": {
"address": "2a0a:2780:4f4c:10::",
"mask": 60
}
}
}
],
"ipv6-prefix-assignment": [
],
"route": [
{
"target": "::",
"mask": 0,
"nexthop": "fe80::ded2:fcff:fe03:782d",
"metric": 512,
"valid": 814,
"source": "2a0a:2780:4f4c::/48"
},
{
"target": "::",
"mask": 0,
"nexthop": "fe80::ded2:fcff:fe03:782d",
"metric": 512,
"valid": 814,
"source": "2a0a:2780:4fff:1dd8:b514:567b:2a52:30d0/128"
}
],
"dns-server": [
"2a0a:2780:2::2",
"2001:4860:4860::8888"
],
"dns-search": [
],
"neighbors": [
],
"inactive": {
"ipv4-address": [
],
"ipv6-address": [
],
"route": [
],
"dns-server": [
],
"dns-search": [
],
"neighbors": [
]
},
"data": {
"passthru": "001700202a0a278000020000000000000000000220014860486000000000000000008888"
}
}
# ifstatus lan
{
"up": true,
"pending": false,
"available": true,
"autostart": true,
"dynamic": false,
"uptime": 2284,
"l3_device": "br-lan",
"proto": "static",
"device": "br-lan",
"updated": [
"addresses"
],
"metric": 0,
"dns_metric": 0,
"delegation": true,
"ipv4-address": [
{
"address": "192.168.8.1",
"mask": 24
}
],
"ipv6-address": [
],
"ipv6-prefix": [
],
"ipv6-prefix-assignment": [
{
"address": "2a0a:2780:4f4c::",
"mask": 60,
"preferred": 1667,
"valid": 3017,
"local-address": {
"address": "2a0a:2780:4f4c::1",
"mask": 60
}
},
{
"address": "fdb6:1bed:70ff::",
"mask": 60,
"local-address": {
"address": "fdb6:1bed:70ff::1",
"mask": 60
}
}
],
"route": [
],
"dns-server": [
],
"dns-search": [
],
"neighbors": [
],
"inactive": {
"ipv4-address": [
],
"ipv6-address": [
],
"route": [
],
"dns-server": [
],
"dns-search": [
],
"neighbors": [
]
},
"data": {
}
}
# ip -6 address show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1000
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
inet6 fe80::724f:57ff:fe8a:755c/64 scope link
valid_lft forever preferred_lft forever
6: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
inet6 2a0a:2780:4f4c::1/60 scope global dynamic
valid_lft 2941sec preferred_lft 1591sec
inet6 fdb6:1bed:70ff::1/60 scope global
valid_lft forever preferred_lft forever
inet6 fe80::724f:57ff:fe8a:755c/64 scope link
valid_lft forever preferred_lft forever
8: eth0.2@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
inet6 2a0a:2780:4fff:1dd8:b514:567b:2a52:30d0/128 scope global dynamic
valid_lft 2941sec preferred_lft 1591sec
inet6 fe80::724f:57ff:fe8a:755d/64 scope link
valid_lft forever preferred_lft forever
9: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
inet6 fe80::724f:57ff:fe8a:755b/64 scope link
valid_lft forever preferred_lft forever
10: wlan1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
inet6 fe80::724f:57ff:fe8a:755c/64 scope link
valid_lft forever preferred_lft forever
# ip -6 route show table all
default from 2a0a:2780:4f4c::/48 via fe80::ded2:fcff:fe03:782d dev eth0.2 metric 512
default from 2a0a:2780:4fff:1dd8:b514:567b:2a52:30d0 via fe80::ded2:fcff:fe03:782d dev eth0.2 metric 512
2a0a:2780:4f4c::/64 dev br-lan metric 1024
unreachable 2a0a:2780:4f4c::/48 dev lo metric 2147483647 error -148
fdb6:1bed:70ff::/64 dev br-lan metric 1024
unreachable fdb6:1bed:70ff::/48 dev lo metric 2147483647 error -148
fe80::/64 dev eth0 metric 256
fe80::/64 dev eth0.2 metric 256
fe80::/64 dev br-lan metric 256
fe80::/64 dev wlan1 metric 256
fe80::/64 dev wlan0 metric 256
local ::1 dev lo table local metric 0
anycast 2a0a:2780:4f4c:: dev br-lan table local metric 0
local 2a0a:2780:4f4c::1 dev br-lan table local metric 0
local 2a0a:2780:4fff:1dd8:b514:567b:2a52:30d0 dev eth0.2 table local metric 0
anycast fdb6:1bed:70ff:: dev br-lan table local metric 0
local fdb6:1bed:70ff::1 dev br-lan table local metric 0
anycast fe80:: dev eth0.2 table local metric 0
anycast fe80:: dev eth0 table local metric 0
anycast fe80:: dev br-lan table local metric 0
anycast fe80:: dev wlan1 table local metric 0
anycast fe80:: dev wlan0 table local metric 0
local fe80::724f:57ff:fe8a:755b dev wlan0 table local metric 0
local fe80::724f:57ff:fe8a:755c dev eth0 table local metric 0
local fe80::724f:57ff:fe8a:755c dev br-lan table local metric 0
local fe80::724f:57ff:fe8a:755c dev wlan1 table local metric 0
local fe80::724f:57ff:fe8a:755d dev eth0.2 table local metric 0
ff00::/8 dev eth0 table local metric 256
ff00::/8 dev br-lan table local metric 256
ff00::/8 dev eth0.2 table local metric 256
ff00::/8 dev wlan1 table local metric 256
ff00::/8 dev wlan0 table local metric 256
# ip -6 rule show
0: from all lookup local
32766: from all lookup main
4200000000: from 2a0a:2780:4f4c::1/60 iif br-lan lookup unspec unreachable
4200000001: from all iif lo lookup unspec 12
4200000006: from all iif br-lan lookup unspec 12
4200000008: from all iif eth0.2 lookup unspec 12
4200000008: from all iif eth0.2 lookup unspec 12