Ping and traceroute failing for eth0.3 on IPv6

Something odd is happening, I can't remember what I might have done to cause this. Ping and traceroute are failing with permission denied error, only for eth0.3 on IPv6.

I have 2 ISPs in mwan3 load balancing. First time I noticed this issue was when mwan3 started reporting wanb6 as offline, I went to diagnose the cause and discovered that ping is failing.

Here's ping under pppoe-wan, note that everything works:

root@EdgeOpenWrt:/# ping -c 2 -I pppoe-wan ipv4.google.com
PING ipv4.google.com (216.58.202.206): 56 data bytes
64 bytes from 216.58.202.206: seq=0 ttl=57 time=18.287 ms
64 bytes from 216.58.202.206: seq=1 ttl=57 time=18.163 ms
--- ipv4.google.com ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 18.163/18.225/18.287 ms

root@EdgeOpenWrt:/# ping -c 2 -I pppoe-wan ipv6.google.com
PING ipv6.google.com (2800:3f0:4001:80f::200e): 56 data bytes
64 bytes from 2800:3f0:4001:80f::200e: seq=0 ttl=55 time=19.853 ms
64 bytes from 2800:3f0:4001:80f::200e: seq=1 ttl=55 time=19.726 ms
--- ipv6.google.com ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 19.726/19.789/19.853 ms

root@EdgeOpenWrt:/# ping -c 2 -I pppoe-wan 172.217.30.110
PING 172.217.30.110 (172.217.30.110): 56 data bytes
64 bytes from 172.217.30.110: seq=0 ttl=57 time=19.402 ms
64 bytes from 172.217.30.110: seq=1 ttl=57 time=19.678 ms
--- 172.217.30.110 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 19.402/19.540/19.678 ms

root@EdgeOpenWrt:/# ping -c 2 -I pppoe-wan 2800:3f0:4001:800::200e
PING 2800:3f0:4001:800::200e (2800:3f0:4001:800::200e): 56 data bytes
64 bytes from 2800:3f0:4001:800::200e: seq=0 ttl=55 time=19.722 ms
64 bytes from 2800:3f0:4001:800::200e: seq=1 ttl=55 time=19.572 ms
--- 2800:3f0:4001:800::200e ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 19.572/19.647/19.722 ms

eth0.3 on IPv4 also works:

root@EdgeOpenWrt:/# ping -c 2 -I eth0.3 ipv4.google.com
PING ipv4.google.com (172.217.29.110): 56 data bytes
64 bytes from 172.217.29.110: seq=0 ttl=54 time=27.025 ms
64 bytes from 172.217.29.110: seq=1 ttl=54 time=28.591 ms
--- ipv4.google.com ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 27.025/27.808/28.591 ms

root@EdgeOpenWrt:/# ping -c 2 -I eth0.3 172.217.30.110
PING 172.217.30.110 (172.217.30.110): 56 data bytes
64 bytes from 172.217.30.110: seq=0 ttl=50 time=32.907 ms
64 bytes from 172.217.30.110: seq=1 ttl=50 time=34.587 ms
--- 172.217.30.110 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 32.907/33.747/34.587 ms

But for IPv6...

root@EdgeOpenWrt:/# ping -c 2 -I eth0.3 ipv6.google.com
PING ipv6.google.com (2800:3f0:4001:800::200e): 56 data bytes
ping: sendto: Permission denied

root@EdgeOpenWrt:/# ping -c 2 -I eth0.3 2800:3f0:4001:800::200e
PING 2800:3f0:4001:800::200e (2800:3f0:4001:800::200e): 56 data bytes
ping: sendto: Permission denied

Any idea what might be happening?

This can be caused by firewall rules DROP-ing packets. Check that with ip6tables-save -t filter

3 Likes

Perhaps I am asking the obvious, but does eth0.3 (still) have an IPv6 address?

1 Like

Thanks a lot! I'm not able to understand its output, I just searched for drop and found nothing...

# Generated by ip6tables-save v1.6.2 on Mon Sep 16 16:17:24 2019
*filter
:INPUT ACCEPT [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
:YAMON34v6 - [0:0]
:YAMON34v6Entry - [0:0]
:YAMON34v6Local - [0:0]
:YAMON34v6_gp_Console - [0:0]
:YAMON34v6_gp_Hardware - [0:0]
:YAMON34v6_gp_ISP - [0:0]
:YAMON34v6_gp_Mobile - [0:0]
:YAMON34v6_gp_NAS - [0:0]
:YAMON34v6_gp_PC - [0:0]
:YAMON34v6_gp_Server - [0:0]
:YAMON34v6_gp_Unknown - [0:0]
:forwarding_lan_rule - [0:0]
:forwarding_rule - [0:0]
:forwarding_wan_rule - [0:0]
:input_lan_rule - [0:0]
:input_rule - [0:0]
:input_wan_rule - [0:0]
:output_lan_rule - [0:0]
:output_rule - [0:0]
:output_wan_rule - [0:0]
:reject - [0:0]
:syn_flood - [0:0]
:zone_lan_dest_ACCEPT - [0:0]
:zone_lan_forward - [0:0]
:zone_lan_input - [0:0]
:zone_lan_output - [0:0]
:zone_lan_src_ACCEPT - [0:0]
:zone_wan_dest_ACCEPT - [0:0]
:zone_wan_dest_REJECT - [0:0]
:zone_wan_forward - [0:0]
:zone_wan_input - [0:0]
:zone_wan_output - [0:0]
:zone_wan_src_REJECT - [0:0]
-A INPUT -j YAMON34v6Entry
-A INPUT -i lo -m comment --comment "!fw3" -j ACCEPT
-A INPUT -m comment --comment "!fw3: Custom input rule chain" -j input_rule
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -m comment --comment "!fw3" -j ACCEPT
-A INPUT -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -m comment --comment "!fw3" -j syn_flood
-A INPUT -i br-lan -m comment --comment "!fw3" -j zone_lan_input
-A INPUT -i eth0.3 -m comment --comment "!fw3" -j zone_wan_input
-A INPUT -i eth0.2 -m comment --comment "!fw3" -j zone_wan_input
-A INPUT -i pppoe-wan -m comment --comment "!fw3" -j zone_wan_input
-A FORWARD -j YAMON34v6Entry
-A FORWARD -m comment --comment "!fw3: Custom forwarding rule chain" -j forwarding_rule
-A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -m comment --comment "!fw3" -j ACCEPT
-A FORWARD -i br-lan -m comment --comment "!fw3" -j zone_lan_forward
-A FORWARD -i eth0.3 -m comment --comment "!fw3" -j zone_wan_forward
-A FORWARD -i eth0.2 -m comment --comment "!fw3" -j zone_wan_forward
-A FORWARD -i pppoe-wan -m comment --comment "!fw3" -j zone_wan_forward
-A FORWARD -m comment --comment "!fw3" -j reject
-A OUTPUT -j YAMON34v6Entry
-A OUTPUT -o lo -m comment --comment "!fw3" -j ACCEPT
-A OUTPUT -m comment --comment "!fw3: Custom output rule chain" -j output_rule
-A OUTPUT -m conntrack --ctstate RELATED,ESTABLISHED -m comment --comment "!fw3" -j ACCEPT
-A OUTPUT -o br-lan -m comment --comment "!fw3" -j zone_lan_output
-A OUTPUT -o eth0.3 -m comment --comment "!fw3" -j zone_wan_output
-A OUTPUT -o eth0.2 -m comment --comment "!fw3" -j zone_wan_output
-A OUTPUT -o pppoe-wan -m comment --comment "!fw3" -j zone_wan_output
-A YAMON34v6 -d 2804:1b2:182:d768::ea5/128 -j YAMON34v6_gp_Server
-A YAMON34v6 -s 2804:1b2:182:d768::ea5/128 -j YAMON34v6_gp_Server
-A YAMON34v6 -d 2804:1b2:182:d768::ea5/128 -j RETURN
-A YAMON34v6 -s 2804:1b2:182:d768::ea5/128 -j RETURN
-A YAMON34v6 -d 2804:1b2:180:a522::ea5/128 -j YAMON34v6_gp_Server
-A YAMON34v6 -s 2804:1b2:180:a522::ea5/128 -j YAMON34v6_gp_Server
-A YAMON34v6 -d 2804:1b2:180:a522::ea5/128 -j RETURN
-A YAMON34v6 -s 2804:1b2:180:a522::ea5/128 -j RETURN
-A YAMON34v6 -d fdfa::2/128 -j YAMON34v6_gp_Server
-A YAMON34v6 -s fdfa::2/128 -j YAMON34v6_gp_Server
-A YAMON34v6 -d fdfa::2/128 -j RETURN
-A YAMON34v6 -s fdfa::2/128 -j RETURN
-A YAMON34v6 -d 2804:1b2:180:1602::ea5/128 -j YAMON34v6_gp_Server
-A YAMON34v6 -s 2804:1b2:180:1602::ea5/128 -j YAMON34v6_gp_Server
-A YAMON34v6 -d 2804:1b2:180:1602::ea5/128 -j RETURN
-A YAMON34v6 -s 2804:1b2:180:1602::ea5/128 -j RETURN
-A YAMON34v6 -d 2804:14c:658b:5fed::ea5/128 -j YAMON34v6_gp_Server
-A YAMON34v6 -s 2804:14c:658b:5fed::ea5/128 -j YAMON34v6_gp_Server
-A YAMON34v6 -d 2804:14c:658b:5fed::ea5/128 -j RETURN
-A YAMON34v6 -s 2804:14c:658b:5fed::ea5/128 -j RETURN
-A YAMON34v6 -d 2804:14c:658b:5fed::100/128 -j YAMON34v6_gp_Server
-A YAMON34v6 -s 2804:14c:658b:5fed::100/128 -j YAMON34v6_gp_Server
-A YAMON34v6 -d 2804:14c:658b:5fed::100/128 -j RETURN
-A YAMON34v6 -s 2804:14c:658b:5fed::100/128 -j RETURN
-A YAMON34v6 -d 2804:14c:658b:5fed::101/128 -j YAMON34v6_gp_PC
-A YAMON34v6 -s 2804:14c:658b:5fed::101/128 -j YAMON34v6_gp_PC
-A YAMON34v6 -d 2804:14c:658b:5fed::101/128 -j RETURN
-A YAMON34v6 -s 2804:14c:658b:5fed::101/128 -j RETURN
-A YAMON34v6 -d 2804:14c:658b:5fed::104/128 -j YAMON34v6_gp_NAS
-A YAMON34v6 -s 2804:14c:658b:5fed::104/128 -j YAMON34v6_gp_NAS
-A YAMON34v6 -d 2804:14c:658b:5fed::104/128 -j RETURN
-A YAMON34v6 -s 2804:14c:658b:5fed::104/128 -j RETURN
-A YAMON34v6 -d 2804:14c:658b:5fed::1/128 -j YAMON34v6_gp_Hardware
-A YAMON34v6 -s 2804:14c:658b:5fed::1/128 -j YAMON34v6_gp_Hardware
-A YAMON34v6 -d 2804:14c:658b:5fed::1/128 -j RETURN
-A YAMON34v6 -s 2804:14c:658b:5fed::1/128 -j RETURN
-A YAMON34v6 -d 2804:14c:658b:1000:45d7:3d14:cfdc:66cd/128 -j YAMON34v6_gp_Hardware
-A YAMON34v6 -s 2804:14c:658b:1000:45d7:3d14:cfdc:66cd/128 -j YAMON34v6_gp_Hardware
-A YAMON34v6 -d 2804:14c:658b:1000:45d7:3d14:cfdc:66cd/128 -j RETURN
-A YAMON34v6 -s 2804:14c:658b:1000:45d7:3d14:cfdc:66cd/128 -j RETURN
-A YAMON34v6 -d fdfa::1/128 -j YAMON34v6_gp_Hardware
-A YAMON34v6 -s fdfa::1/128 -j YAMON34v6_gp_Hardware
-A YAMON34v6 -d fdfa::1/128 -j RETURN
-A YAMON34v6 -s fdfa::1/128 -j RETURN
-A YAMON34v6 -d 2804:1b2:180:a522::1/128 -j YAMON34v6_gp_Hardware
-A YAMON34v6 -s 2804:1b2:180:a522::1/128 -j YAMON34v6_gp_Hardware
-A YAMON34v6 -d 2804:1b2:180:a522::1/128 -j RETURN
-A YAMON34v6 -s 2804:1b2:180:a522::1/128 -j RETURN
-A YAMON34v6 -d 2804:1b2:180:1602::1/128 -j YAMON34v6_gp_Hardware
-A YAMON34v6 -s 2804:1b2:180:1602::1/128 -j YAMON34v6_gp_Hardware
-A YAMON34v6 -d 2804:1b2:180:1602::1/128 -j RETURN
-A YAMON34v6 -s 2804:1b2:180:1602::1/128 -j RETURN
-A YAMON34v6 -j RETURN
-A YAMON34v6Entry -s fe00::/7 -d fe00::/7 -j YAMON34v6Local
-A YAMON34v6Entry -s fe00::/7 -d fe00::/7 -j RETURN
-A YAMON34v6Entry -s fe00::/7 -d fc00::/7 -j YAMON34v6Local
-A YAMON34v6Entry -s fe00::/7 -d fc00::/7 -j RETURN
-A YAMON34v6Entry -s fc00::/7 -d fe00::/7 -j YAMON34v6Local
-A YAMON34v6Entry -s fc00::/7 -d fe00::/7 -j RETURN
-A YAMON34v6Entry -s fc00::/7 -d fc00::/7 -j YAMON34v6Local
-A YAMON34v6Entry -s fc00::/7 -d fc00::/7 -j RETURN
-A YAMON34v6Entry -j YAMON34v6
-A YAMON34v6Local -j RETURN
-A YAMON34v6_gp_Console -j RETURN
-A YAMON34v6_gp_Hardware -j RETURN
-A YAMON34v6_gp_ISP -j RETURN
-A YAMON34v6_gp_Mobile -j RETURN
-A YAMON34v6_gp_NAS -j RETURN
-A YAMON34v6_gp_PC -j RETURN
-A YAMON34v6_gp_Server -j RETURN
-A YAMON34v6_gp_Unknown -j RETURN
-A reject -p tcp -m comment --comment "!fw3" -j REJECT --reject-with tcp-reset
-A reject -m comment --comment "!fw3" -j REJECT --reject-with icmp6-port-unreachable
-A syn_flood -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -m limit --limit 25/sec --limit-burst 50 -m comment --comment "!fw3" -j RETURN
-A syn_flood -m comment --comment "!fw3" -j DROP
-A zone_lan_dest_ACCEPT -o br-lan -m comment --comment "!fw3" -j ACCEPT
-A zone_lan_forward -m comment --comment "!fw3: Custom lan forwarding rule chain" -j forwarding_lan_rule
-A zone_lan_forward -m comment --comment "!fw3: Zone lan to wan forwarding policy" -j zone_wan_dest_ACCEPT
-A zone_lan_forward -m comment --comment "!fw3" -j zone_lan_dest_ACCEPT
-A zone_lan_input -m comment --comment "!fw3: Custom lan input rule chain" -j input_lan_rule
-A zone_lan_input -m comment --comment "!fw3" -j zone_lan_src_ACCEPT
-A zone_lan_output -m comment --comment "!fw3: Custom lan output rule chain" -j output_lan_rule
-A zone_lan_output -m comment --comment "!fw3" -j zone_lan_dest_ACCEPT
-A zone_lan_src_ACCEPT -i br-lan -m conntrack --ctstate NEW,UNTRACKED -m comment --comment "!fw3" -j ACCEPT
-A zone_wan_dest_ACCEPT -o eth0.3 -m conntrack --ctstate INVALID -m comment --comment "!fw3: Prevent NAT leakage" -j DROP
-A zone_wan_dest_ACCEPT -o eth0.3 -m comment --comment "!fw3" -j ACCEPT
-A zone_wan_dest_ACCEPT -o eth0.2 -m conntrack --ctstate INVALID -m comment --comment "!fw3: Prevent NAT leakage" -j DROP
-A zone_wan_dest_ACCEPT -o eth0.2 -m comment --comment "!fw3" -j ACCEPT
-A zone_wan_dest_ACCEPT -o pppoe-wan -m conntrack --ctstate INVALID -m comment --comment "!fw3: Prevent NAT leakage" -j DROP
-A zone_wan_dest_ACCEPT -o pppoe-wan -m comment --comment "!fw3" -j ACCEPT
-A zone_wan_dest_REJECT -o eth0.3 -m comment --comment "!fw3" -j reject
-A zone_wan_dest_REJECT -o eth0.2 -m comment --comment "!fw3" -j reject
-A zone_wan_dest_REJECT -o pppoe-wan -m comment --comment "!fw3" -j reject
-A zone_wan_forward -m comment --comment "!fw3: Custom wan forwarding rule chain" -j forwarding_wan_rule
-A zone_wan_forward -p ipv6-icmp -m icmp6 --icmpv6-type 128 -m limit --limit 1000/sec -m comment --comment "!fw3: Allow-ICMPv6-Forward" -j ACCEPT
-A zone_wan_forward -p ipv6-icmp -m icmp6 --icmpv6-type 129 -m limit --limit 1000/sec -m comment --comment "!fw3: Allow-ICMPv6-Forward" -j ACCEPT
-A zone_wan_forward -p ipv6-icmp -m icmp6 --icmpv6-type 1 -m limit --limit 1000/sec -m comment --comment "!fw3: Allow-ICMPv6-Forward" -j ACCEPT
-A zone_wan_forward -p ipv6-icmp -m icmp6 --icmpv6-type 2 -m limit --limit 1000/sec -m comment --comment "!fw3: Allow-ICMPv6-Forward" -j ACCEPT
-A zone_wan_forward -p ipv6-icmp -m icmp6 --icmpv6-type 3 -m limit --limit 1000/sec -m comment --comment "!fw3: Allow-ICMPv6-Forward" -j ACCEPT
-A zone_wan_forward -p ipv6-icmp -m icmp6 --icmpv6-type 4/0 -m limit --limit 1000/sec -m comment --comment "!fw3: Allow-ICMPv6-Forward" -j ACCEPT
-A zone_wan_forward -p ipv6-icmp -m icmp6 --icmpv6-type 4/1 -m limit --limit 1000/sec -m comment --comment "!fw3: Allow-ICMPv6-Forward" -j ACCEPT
-A zone_wan_forward -p esp -m comment --comment "!fw3: Allow-IPSec-ESP" -j zone_lan_dest_ACCEPT
-A zone_wan_forward -p udp -m udp --dport 500 -m comment --comment "!fw3: Allow-ISAKMP" -j zone_lan_dest_ACCEPT
-A zone_wan_forward -m comment --comment "!fw3" -j zone_wan_dest_REJECT
-A zone_wan_input -m comment --comment "!fw3: Custom wan input rule chain" -j input_wan_rule
-A zone_wan_input -s fc00::/6 -d fc00::/6 -p udp -m udp --dport 546 -m comment --comment "!fw3: Allow-DHCPv6" -j ACCEPT
-A zone_wan_input -s fe80::/10 -p ipv6-icmp -m icmp6 --icmpv6-type 130/0 -m comment --comment "!fw3: Allow-MLD" -j ACCEPT
-A zone_wan_input -s fe80::/10 -p ipv6-icmp -m icmp6 --icmpv6-type 131/0 -m comment --comment "!fw3: Allow-MLD" -j ACCEPT
-A zone_wan_input -s fe80::/10 -p ipv6-icmp -m icmp6 --icmpv6-type 132/0 -m comment --comment "!fw3: Allow-MLD" -j ACCEPT
-A zone_wan_input -s fe80::/10 -p ipv6-icmp -m icmp6 --icmpv6-type 143/0 -m comment --comment "!fw3: Allow-MLD" -j ACCEPT
-A zone_wan_input -p ipv6-icmp -m icmp6 --icmpv6-type 128 -m limit --limit 1000/sec -m comment --comment "!fw3: Allow-ICMPv6-Input" -j ACCEPT
-A zone_wan_input -p ipv6-icmp -m icmp6 --icmpv6-type 129 -m limit --limit 1000/sec -m comment --comment "!fw3: Allow-ICMPv6-Input" -j ACCEPT
-A zone_wan_input -p ipv6-icmp -m icmp6 --icmpv6-type 1 -m limit --limit 1000/sec -m comment --comment "!fw3: Allow-ICMPv6-Input" -j ACCEPT
-A zone_wan_input -p ipv6-icmp -m icmp6 --icmpv6-type 2 -m limit --limit 1000/sec -m comment --comment "!fw3: Allow-ICMPv6-Input" -j ACCEPT
-A zone_wan_input -p ipv6-icmp -m icmp6 --icmpv6-type 3 -m limit --limit 1000/sec -m comment --comment "!fw3: Allow-ICMPv6-Input" -j ACCEPT
-A zone_wan_input -p ipv6-icmp -m icmp6 --icmpv6-type 4/0 -m limit --limit 1000/sec -m comment --comment "!fw3: Allow-ICMPv6-Input" -j ACCEPT
-A zone_wan_input -p ipv6-icmp -m icmp6 --icmpv6-type 4/1 -m limit --limit 1000/sec -m comment --comment "!fw3: Allow-ICMPv6-Input" -j ACCEPT
-A zone_wan_input -p ipv6-icmp -m icmp6 --icmpv6-type 133 -m limit --limit 1000/sec -m comment --comment "!fw3: Allow-ICMPv6-Input" -j ACCEPT
-A zone_wan_input -p ipv6-icmp -m icmp6 --icmpv6-type 135 -m limit --limit 1000/sec -m comment --comment "!fw3: Allow-ICMPv6-Input" -j ACCEPT
-A zone_wan_input -p ipv6-icmp -m icmp6 --icmpv6-type 134 -m limit --limit 1000/sec -m comment --comment "!fw3: Allow-ICMPv6-Input" -j ACCEPT
-A zone_wan_input -p ipv6-icmp -m icmp6 --icmpv6-type 136 -m limit --limit 1000/sec -m comment --comment "!fw3: Allow-ICMPv6-Input" -j ACCEPT
-A zone_wan_input -m comment --comment "!fw3" -j zone_wan_src_REJECT
-A zone_wan_output -m comment --comment "!fw3: Custom wan output rule chain" -j output_wan_rule
-A zone_wan_output -m comment --comment "!fw3" -j zone_wan_dest_ACCEPT
-A zone_wan_src_REJECT -i eth0.3 -m comment --comment "!fw3" -j reject
-A zone_wan_src_REJECT -i eth0.2 -m comment --comment "!fw3" -j reject
-A zone_wan_src_REJECT -i pppoe-wan -m comment --comment "!fw3" -j reject
COMMIT
# Completed on Mon Sep 16 16:17:24 2019

Thanks too!

According to Luci's network page, wanb6 does have a IPv6 address and PD. IDK how to verify that by CLI.

After tried it a bit on my device, it seems to me this can also be caused by route configuration.

My device does not have IPv6 connection, ping 2800:3f0:4001:800::200e resulted in Permission denied right away.

Then I added a route making it through br-lan instead of eth0.3 with

ip -6 route add 2800:3f0:4001:800::200e/128 dev br-lan

Ping without specifying device will timeout (expected) but ping -I eth0.3 will fail with Permission denied.

It seems routing subsystem of IPv6 and IPv4 is very different in this regard.

Maybe you can confirm that with the following commands

# check route to 2800:3f0:4001:800::200e
ip -6 route
ip -6 route get 2800:3f0:4001:800::200e

# route 2800:... through eth0.3
ip -6 route add 2800:3f0:4001:800::200e/128 dev eth0.3

# try ping it through eth0.3 again
ping -I eth0.3 2800:3f0:4001:800::200e
3 Likes
# This doesn't work
$ ip -6 route get 1:: dev ${NET_IF}
RTNETLINK answers: Permission denied

# This works
$ ip -6 route get 1:: dev ${NET_IF} from ${NET_ADDR}
1 Like

@yousong I don't really know what you did, but it worked!

Any other IPv6 host I try to ping I get permission denied error, but 2800:3f0:4001:800::200e works!

root@EdgeOpenWrt:/# ping -c 2 -I eth0.3 ipv6.google.com
PING ipv6.google.com (2800:3f0:4004:800::200e): 56 data bytes
ping: sendto: Permission denied

root@EdgeOpenWrt:/# ping -c 2 -I eth0.3 2800:3f0:4001:800::200e
PING 2800:3f0:4001:800::200e (2800:3f0:4001:800::200e): 56 data bytes
64 bytes from 2800:3f0:4001:800::200e: seq=0 ttl=49 time=40.800 ms
64 bytes from 2800:3f0:4001:800::200e: seq=1 ttl=49 time=44.981 ms
--- 2800:3f0:4001:800::200e ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 40.800/42.890/44.981 ms

Sadly I don't understand ip command and ip6tables. I tested and pppoe-wan remains working.

@vgaetera thanks! I also wasn't able to understand your test. I'll read that link.

What might be wrong on my router? I didn't manually change any routes.

1 Like

It's likely a source routing issue.
Post your persistent and runtime network configuration:

uci show network
ip -6 a; ip -6 r s t all; ip -6 ru
1 Like

I'm over an hour trying to post and it's failing, IDK why.

I placed the output on https://pastebin.com/RLSnjR16

2 Likes

Try this way:

ip -6 route get 1::
ip -6 route get 1:: dev pppoe-wan
ip -6 route get 1:: dev pppoe-wan from 2804:1b2:206:520:a9f2:ca03:4dec:7c71
ip -6 route get 1:: dev eth0.3
ip -6 route get 1:: dev eth0.3 from 2804:14c:658b:1000:45d7:3d14:cfdc:66cd
ping6 -c 3 example.org
ping6 -c 3 -I 2804:1b2:206:520:a9f2:ca03:4dec:7c71 example.org
ping6 -c 3 -I 2804:14c:658b:1000:45d7:3d14:cfdc:66cd example.org

It seems that in some scenarios OpenWrt can't correctly select the source IPv6 address for routing.

I have a similar bug report here:
https://bugs.openwrt.org/index.php?do=details&task_id=2167

The only workaround is to add another IPv6 default route without source address filter:
https://openwrt.org/docs/guide-user/network/ipv6/ipv6_henet#default_route

2 Likes

This didn't seem promissing...

root@EdgeOpenWrt:/# ip -6 route get 1::
RTNETLINK answers: Permission denied
root@EdgeOpenWrt:/# ip -6 route get 1:: dev pppoe-wan
RTNETLINK answers: Permission denied
root@EdgeOpenWrt:/# ip -6 route get 1:: dev pppoe-wan from 2804:1b2:206:520:a9f2
:ca03:4dec:7c71
1:: from 2804:1b2:206:520:a9f2:ca03:4dec:7c71 via fe80::6b0:e7ff:fec9:4557 dev pppoe-wan proto static src 2804:1b2:206:520:a9f2:ca03:4dec:7c71 metric 512 pref medium
root@EdgeOpenWrt:/# ip -6 route get 1:: dev eth0.3
RTNETLINK answers: Permission denied
root@EdgeOpenWrt:/# ip -6 route get 1:: dev eth0.3 from 2804:14c:658b:1000:45d7:
3d14:cfdc:66cd
1:: from 2804:14c:658b:1000:45d7:3d14:cfdc:66cd via fe80::201:5cff:fe66:7646 dev eth0.3 proto static src 2804:14c:658b:1000:45d7:3d14:cfdc:66cd metric 512 pref medium
root@EdgeOpenWrt:/# ping6 -c 3 openwrt.org
PING openwrt.org (2a03:b0c0:3:d0::1af1:1): 56 data bytes
64 bytes from 2a03:b0c0:3:d0::1af1:1: seq=0 ttl=52 time=238.547 ms
64 bytes from 2a03:b0c0:3:d0::1af1:1: seq=1 ttl=52 time=227.981 ms
64 bytes from 2a03:b0c0:3:d0::1af1:1: seq=2 ttl=52 time=228.015 ms

--- openwrt.org ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 227.981/231.514/238.547 ms
root@EdgeOpenWrt:/# ping6 -c 3 -I 2804:1b2:206:520:a9f2:ca03:4dec:7c71 openwrt.o
rg
PING openwrt.org (2a03:b0c0:3:d0::1af1:1) from 2804:1b2:206:520:a9f2:ca03:4dec:7c71: 56 data bytes
64 bytes from 2a03:b0c0:3:d0::1af1:1: seq=0 ttl=52 time=231.257 ms
64 bytes from 2a03:b0c0:3:d0::1af1:1: seq=1 ttl=52 time=230.968 ms
64 bytes from 2a03:b0c0:3:d0::1af1:1: seq=2 ttl=52 time=231.466 ms

--- openwrt.org ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 230.968/231.230/231.466 ms
root@EdgeOpenWrt:/# ping6 -c 3 -I 2804:14c:658b:1000:45d7:3d14:cfdc:66cd openwrt
.org
PING openwrt.org (2a03:b0c0:3:d0::1af1:1) from 2804:14c:658b:1000:45d7:3d14:cfdc:66cd: 56 data bytes
64 bytes from 2a03:b0c0:3:d0::1af1:1: seq=0 ttl=46 time=232.976 ms
64 bytes from 2a03:b0c0:3:d0::1af1:1: seq=1 ttl=46 time=235.716 ms
64 bytes from 2a03:b0c0:3:d0::1af1:1: seq=2 ttl=46 time=232.910 ms

--- openwrt.org ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 232.910/233.867/235.716 ms

Won't this route you suggested mess with mwan3? Now I remembered, it's reporting warnings with this interface too. Could you take a look before I add this route? Maybe it's the cause of this permission issue.

If this is caused by some problem on routing, I think permission denied is a misleading error message. It implies that root isn't permitted to run this command, not that it's a routing problem. Should I add a bug report suggesting a better message?

Actually, is works exactly as I expected.

So, if you want to use ping6 via specific interface, you need to specify the source address.

The reason is here:

About mwan3 I'm not sure as I don't use it, however it works the same way as the one you added earlier:

2 Likes

Thanks a lot to taking the time to help me understand this.

IDK how it handles routes, but for this matter it uses ping to verify if a WAN/interface is reaching internet and can be used for load balancing and fail over. It relies on interface name for that, so it's not working because passing the interface name as parameter isn't working.

What's odd is that it was all working before. I'm not sure, but I believe it started when I moved this modem to bridge and changed network settings to PPPoE connect.

Anyway, so if I don't specify an interface to where the ping must use, as in ping -c 2 ipv6.google.com, then OpenWRT just chooses the route randomly or by mwan3 rules.

But if I need to use a specific interface as in ping -c 2 -I eth0.3 ipv6.google.com, then it doesn't work with permission denied message.

Indeed, if I use ping -c 2 -I 2804:14c:658b:5fed::1 ipv6.google.com (ISP1 prefix + "suffix" [config interface lan option ip6ifaceid]) or ping -c 2 -I 2804:1b2:180:afc1::1 ipv6.google.com (ISP2 prefix), both works. The same is valid for traceroute -i.

What I don't understand is why ping -c 2 -I eth0.3 ipv6.google.com doesn't work but ping -c 2 -I pppoe-wan ipv6.google.com does.

The problem is that these global prefixes are dynamic. And my main use for this is precisely mwan3 pinging to verify if each WAN/interface is up and can be used.

Other than ping, it offers arping and httping to do the job. I'm gonna read about them before testing on my router.

But, isn't there a stable way to test if a WAN/interface is reaching internet, that can be used in scripts?

ddns is modular and allows us to use custom script to find out whatever IP address we wanna attribute for a domain name. mwan3 doesn't seem to have such feature. If needed be, I could add a feature request for that. Sadly I'm not good on sh/bash/python to help develop it :confused:

1 Like

Try to compare these:

ip -6 r s t all dev eth0.3
ip -6 r s t all dev pppoe-wan

The output should be symmetrical.

1 Like
root@EdgeOpenWrt:/# ip -6 r s t all dev pppoe-wan
default via fe80::6b0:e7ff:fec9:4557 table 2 metric 1024 pref medium
default from 2804:1b2:180:afc1::/64 via fe80::6b0:e7ff:fec9:4557 proto static metric 512 pref medium
default from 2804:1b2:206:2174::/64 via fe80::6b0:e7ff:fec9:4557 proto static metric 512 pref medium
2804:1b2:206:2174::/64 proto static metric 256 pref medium
fe80::/10 metric 1 pref medium
fe80::/10 proto kernel metric 256 pref medium
anycast 2804:1b2:206:2174:: table local proto kernel metric 0 pref medium
local 2804:1b2:206:2174:6d97:138e:9bf8:fc30 table local proto kernel metric 0 pref medium
anycast fe80:: table local proto kernel metric 0 pref medium
local fe80::6d97:138e:9bf8:fc30 table local proto kernel metric 0 pref medium
ff00::/8 table local metric 256 pref medium
root@EdgeOpenWrt:/# ip -6 r s t all dev eth0.3
default via fe80::201:5cff:fe66:7646 table 4 metric 1024 pref medium
default from 2804:14c:658b:1000:45d7:3d14:cfdc:66cd via fe80::201:5cff:fe66:7646 proto static metric 512 pref medium
default from 2804:14c:658b:5fed::/64 via fe80::201:5cff:fe66:7646 proto static metric 512 pref medium
2800:3f0:4001:800::200e metric 1024 pref medium
fe80::/64 proto kernel metric 256 pref medium
local 2804:14c:658b:1000:45d7:3d14:cfdc:66cd table local proto kernel metric 0 pref medium
anycast fe80:: table local proto kernel metric 0 pref medium
local fe80::822a:a8ff:fe5d:79d7 table local proto kernel metric 0 pref medium
ff00::/8 table local metric 256 pref medium

Yeah they aren't exactly symmetrical :confused: But then one is PPPoE and the other is DHCPv6.

2 Likes

I checked on my OpenWrt with the routing configuration as close to yours as possible and it still works, so the problem might not be related to routing directly.

Try to temporary stop the firewall service:

/etc/init.d/firewall stop
ping6 -c 3 -I pppoe-wan example.org
ping6 -c 3 -I eth0.3 example.org
/etc/init.d/firewall start
1 Like

It took me a while to find out the root cause of "Permission defined" (EACCES). They should be caused by iif lo failed_policy rule installed by OpenWrt patches

  • target/linux/generic/pending-4.19/670-ipv6-allow-rejecting-with-source-address-failed-policy.patch
  • package/network/utils/iproute2/patches/180-drop_FAILED_POLICY.patch

My current understanding of the situation is that when doing source address selection

  • Kernel does route lookup to find output device. This will fail because no route matches. Those default routes in main table are all constrained by "from" directive. Those default routes in table 2 and 4 are guarded by "from" directive in ip-rules
  • With output device unspecified, "Rule 5: Prefer outgoing interface." of rfc3484 will not apply. "Rule 8: Use longest matching prefix. " results in tie for destination 2800:3f0:4001:800::200e. The source address selection algorithm is free to use scope global addresses from other interfaces, e.g. pppoe-wan, etc.

If the above assumption holds, after source address selection, we end up with fl6.saddr from "pppoe-wan" and fl6.flowi6_oif being "eth0.3" (SO_BINDTODEVICE).

Then kernel does route lookup again to find output route, it fails again hitting iif lo failed_policy. And this time the EACCES will be returned to userspace.

My conclusion is that source address constrained routes in main table should be the cause. However I am not sure why they are there and preferred over the traditional destination based routes.

There are a few ways to check and confirm the level of correctness the above words are.

# ping another address so that "Rule 8:  Use longest matching prefix. "
# prefers "2804:14c:658b:1000:45d7:3d14:cfdc:66cd/128" from eth0.3 .
#
# Timeout is not a concern we just want to make sense the 
# "Permission denied" error.
ping -I eth0.3 2804:14...
ping -I pppoe-wan 2804:14...

# remove those "default from xxx" routes from main table 
# and replace them with traditional ones.
ip -6 route del default from 2804:14c:658b:1000:45d7:3d14:cfdc:66cd via fe80::201:5cff:fe66:7646 dev eth0.3 proto static metric 512 pref medium
ip -6 route del default from 2804:14c:658b:5fed::/64 via fe80::201:5cff:fe66:7646 dev eth0.3 proto static metric 512 pref medium
ip -6 route del default from 2804:1b2:182:d768::/64 via fe80::6b0:e7ff:fec9:4557 dev pppoe-wan proto static metric 512 pref medium
ip -6 route del default from 2804:1b2:206:520::/64 via fe80::6b0:e7ff:fec9:4557 dev pppoe-wan proto static metric 512 pref medium
ip -6 route add default via fe80::201:5cff:fe66:7646 dev eth0.3 proto static metric 512 pref medium
ip -6 route add default via fe80::201:5cff:fe66:7646 dev eth0.3 proto static metric 512 pref medium
ip -6 route add default via fe80::6b0:e7ff:fec9:4557 dev pppoe-wan proto static metric 512 pref medium
ip -6 route add default via fe80::6b0:e7ff:fec9:4557 dev pppoe-wan proto static metric 512 pref medium
ping -c 2 -I pppoe-wan ipv6.google.com
ping -c 2 -I eth0.3 ipv6.google.com
3 Likes

Did you ever get to the bottom of this?

It's worth updating this that @brianjmurrell recently committed a fix to mwan3 which was merged which resolves ping issues with IPv6 interfaces. The issue appears to be OpenWrt using the wrong source address in some cases. This leads to mwan3 stating the interface as down when it's not.

The change in mwan3 2.8.3-2 and newer makes track_ip ping the interface with the source address obtained from the interface which is more reliable, than the interface name because of the mentioned source address issue.

There are also differences when using the busybox ping6 vs iputils-ping6 packages, which I think is all down to the source address problem.

1 Like

My system is:
Model: TP-Link Archer C7 v5
Architecture: Qualcomm Atheros QCA956X ver 1 rev 0
Firmware Version: OpenWrt 19.07.2 r10947-65030d81f3 / LuCI openwrt-19.07 branch git-20.064.69776-e8c638c
Kernel Version: 4.14.171

I am just using PPPoE and thats all. I seem to have same problem.

root@OpenWrt:~# ping 2800:3f0:4001:800::200e
PING 2800:3f0:4001:800::200e (2800:3f0:4001:800::200e): 56 data bytes
ping: sendto: Permission denied
root@OpenWrt:~#

I did not install any mwan3 package. Though I see mwan3 2.8.4-2 is available in LUCI.

So, how one should be solving that problem? Simply by installing that package?

Thanks.

1 Like