I recently setup another WAN connection by assigning one of my existing LAN ports on my Linksys WRT3200ACM onto a new VLAN (eth0.3), I have both working IPv4 and IPv6 connectivity on the eth0.3 interface confirmed through ping and traceroute, however I have noticed some IPv6 domains/literal IPv6 addresses return the following error:
ping6: sendto: Permission denied
The same with traceroute and even cURL (error 7), however it's not happening on all IPv6 DNS/literal tests.
root@linksys-wrt3200acm:~# ping6 -I eth0.3 ipv6.google.com
PING ipv6.google.com (2a00:1450:4009:816::200e): 56 data bytes
64 bytes from 2a00:1450:4009:816::200e: seq=0 ttl=50 time=123.716 ms
64 bytes from 2a00:1450:4009:816::200e: seq=1 ttl=50 time=28.521 ms
64 bytes from 2a00:1450:4009:816::200e: seq=2 ttl=50 time=24.645 ms
64 bytes from 2a00:1450:4009:816::200e: seq=3 ttl=50 time=26.934 ms
64 bytes from 2a00:1450:4009:816::200e: seq=4 ttl=50 time=25.919 ms
Yet if I then test with one the Google IPv6 DNS addresses:
I first thought firewall, but I'm not sure because I figured all IPv6 traffic wouldn't work at all, yet it does, just I found this example and a few others. It was brought to my attention as mwan3 was detecting the interface as down, which led me to discover ping test were failing as the interface ping test was using Google IPv6 DNS and OpenDNS, both which return the above error when pinging.
Can anyone shed some light on the the ping error is indicating?
After investigating a bit further it appears to be when using the interface name, if I specify the source IPv6 address on the eth0.3 interface, ping, traceroute and cURL all work fine, so perhaps the wrong address is be used, although it's weird how it works for other IPv6 domains without doing this.
root@linksys-wrt3200acm:~# ping6 -I 2a01:4c8:c40:xxx:xxxx:xxxx:x:xxx 2001:4860:4860::8888
PING 2001:4860:4860::8888 (2001:4860:4860::8888) from 2a01:4c8:c40:xxxx:xxxx:xxxx:x:xxx: 56 data bytes
64 bytes from 2001:4860:4860::8888: seq=0 ttl=50 time=148.422 ms
64 bytes from 2001:4860:4860::8888: seq=1 ttl=50 time=28.466 ms
64 bytes from 2001:4860:4860::8888: seq=2 ttl=50 time=24.052 ms
64 bytes from 2001:4860:4860::8888: seq=3 ttl=50 time=28.513 ms
...it doesn't use the interface which has this IP, only uses the provided IP as source.
Please post here the output of the following commands, copy and paste the whole block:
uci export network; uci export wireless; \
uci export dhcp; uci export firewall; uci export mwan3; \
head -n -0 /etc/firewall.user; \
ip -4 addr ; ip -4 ro ls tab all ; ip -4 ru; \
ip -6 addr ; ip -6 ro ls tab all ; ip -6 ru; \
ls -l /etc/resolv.* /tmp/resolv.*; head -n -0 /etc/resolv.* /tmp/resolv.*
Here's the output as it does contain various bits of sensitive information I have redacted parts, it's too big to post here, so here's a link on Pastebin:
The resolv.conf and the other files in /tmp are indeed empty? Are you able to query them without packet loss or going out of the wrong interface?
The mwan3 configuration doesn't have any ipv6 configuration except the interfaces. I have in every policy a member of each address family and the default rule doesn't specify addresses.
They are. I have peerdns disabled and resolv conf ignored in dnsmasq, the default DNS lookup on the router would be itself 127.0.0.1, the DNS chain on the LAN is pushed through DHCP, but DNS works fine.
I haven't actually started using mwan3 for IPv6 yet, given the IPv6 routing table can at least handle that for now, WAN6 is the default, which works fine. I am currently looking to implementing IPv6 into the configuration, I can see the default config has IPv6 examples which I'll look at soon:
I don't understand though why some IPv6 domains/addresses return permission denied, while others don't. Would it suggest a routing problem somewhere?
I had the same issues more or less with mwan3.
Since I have a single wan6 I thought it wouldn't be necessary to configure it in mwan3, but I had erratic behaviour until I configured it properly.
You can test it with mwan3 disabled to verify.
Interesting you say that, as I have been doing some IPv6 related testing with mwan3 and it does seem to be causing issues in some cases, mainly it breaks connectivity with 6in4, although certain combinations of restarting the interface seem to bring it back.
My WAN6 is actually 6in4 from HE, I then have WAN2 which has both IPv4 and IPv6 but in order for IPv6 to work on WAN2 I need to have a virtual interface WAN2_6 which uses DHCPv6 to obtain a prefix, that also works with the exception of the weird sendto: permissioned denied errors only on some IPv6 request though, although that's specific to this interface, the 6in4 interface doesn't have those issues.
I'm a little confused about the mwan3 config though, the default rule uses 0.0.0.0/0 to me that's IPv4 address family only? Yet mwan3 seems to add a default ::/0 rule when looking at the mwan3 status output, even though no specific IPv6 rule is specified for this. I assume it's tied to the interface family value.
Maybe for now it would be better to disable any IPv6 members or interfaces so no default rule is added in MWAN3 and let the IPv6 routing table handle it.
You can try that, but if I recall correctly that didn't make much of a difference for me.
The guideline of the example is what I use now and it works fine.
Thanks for the tip. I'll look at implementing IPv6 into mwan3, it is something I was looking into, The default config isn't actually too far my setup with two WAN with v6 anyway. I'll see if it makes a difference.
Unfortunately, like other similar reports, it appears mwan3 is breaking 6in4 connectivity, if I disable it completely, everything is fine. The interface is configured in the config but is reporting as down according to the mwan3 status (because ping track_ip is failing). Although for a short period of time, the interface appear up.
Yeah sorry, I set an expiry to make sure I didn't leak my entire network config for all to see. I can re-post it again, if it would be helpful. Although I'm starting to think the original ping6 problem with the second WAN isn't related to mwan3 as it seems to happen regardless if the service is running or not.
I have this route configured to make sure the router uses the 6in4-wan6 interface as the default IPv6 gateway:
Likewise I also have to set a static IPv4 route to the HE.net IPv4 tunnel endpoint, as it seems with two WANs a static route to the HE tunnel endpoints gets created on the wrong WAN automatically, so by creating the static route this makes sure it goes through the right WAN, even though the route might be added to the second WAN.
If you remove all sensitive data like usernames, passwords, keys, mac there is nothing to leak and can serve as base configuration for others with the same problem.
The route6 is not necessary as you are getting the default gateway from both 6in4 and wan2_6. If you want to give priority to one of them, you can change the metric.
The route for he[.]net server is also pointless because with mwan3 alternative routing tables are used, so the main routing table is not looked up by default. What you need is a policy in mwan3 for the IP of he[.]net server.
Unfortunately, with that rule in place mwan3 still breaks 6in4, this was the rule generated based on the iptables mangle table:
0 0 mwan3_policy_wan_only all -- * * 0.0.0.0/0 216.66.80.26 mark match 0x0/0x3f00 /* he_net */
Interestingly though no bytes recorded, so it doesn't seem like the rule it being hit. Given the known problems I might just have to give up on trying to shape 6in4 with mwan3.
Thanks, but it's still the same and no packets being received on that rule, so proto 41 doesn't seem to be marking it this far.
0 0 mwan3_policy_wan_only 41 -- * * 0.0.0.0/0 216.66.80.26 mark match 0x0/0x3f00 /* he_net *
Having another SSH connection open when restarting mwan3 I can see ping responses to the 6in4-wan6 interface happen very briefly I assume before the mwan3 routing/firewall rules are implemented and then it's dead. If stop mwan3 6in4-wan6 ping works.