HE IPv6 Tunnel + Mwan3 still completely broken

Installing and Using OpenWrt Network and Wireless Configuration
1

Starting point:
pppoe-wan and lte-wan (usb in hilink mode appearing as eth1) as well as 6in4-henet tunnel interface

mwan failover rule based on source 0.0.0.0/0 dest 0.0.0.0/0
first problem Mwan3 config not working at all this way, you have to set source or dest to something else than 0.0.0.0 or otherwise you automatically get Active ipv6 user rules although never defined.

second problem
6in4-henet always gets its local address set to the eth1 address, disobeys current mwan routing which is normally pppoe-wan (as long as there is no failover). Only (known to me) way to change that is to enter the current pppoe-wan address in the 6in4-henet network config, but that's not sustainable as it is dynamic

third problem
6in4-henet also adds a default route to the tunnel endpoint via the (wrongly) selected interface

So in order to make mwan3 AND 6in4 work you have to:

  • set mwan3 failover rules source to 192.168.0.0/16 or something that still suites you
  • manually enter current (main) ip address to 6in4-henet device config and restart device
  • manually delete wrong route via wrong wan device AND add a correct one

Point 1 is not that bad, but 2 and 3 are really a blocker.
Any idea how to script points 2 and 3 until there is a proper fix for mwan+6in4?
Even a solution that always select pppoe-wans address would be ok for the moment

for now I added the following quick fix to /etc/mwan3.user
X.X.X.X is tunnel endpoint.
of course this is only working as long as lte failover is inactive (pppoe-wan) healty

ip link set 6in4-henet type sit local $(ip -o address show dev pppoe-wan|cut -f7 -d" ")
ip route del X.X.X.X
ip route add X.X.X.X dev pppoe-wan

Old post, obsolete

My configuration is pretty standard, pppoe-wan xDSL has only IPv4 connectivity, so I added 6in4-henet tunnel as described in the wiki. Everything worked fine for months now. No special firewall rules had to be added.

Now suddenly IPv6 is not routable any more.

my routing table looks like this:

default from 2001:470:1f0a:af::/64 dev 6in4-henet proto static metric 1024 pref medium
default from 2001:470:7059::/48 dev 6in4-henet proto static metric 1024 pref medium
2001:470:1f0a:af::/64 dev 6in4-henet proto kernel metric 256 pref medium
2001:470:7059::/64 dev br-lan proto static metric 1024 pref medium
unreachable 2001:470:7059::/48 dev lo proto static metric 2147483647 error -148 pref medium
fd00:0:0:64::/64 dev br-lan proto static metric 1024 pref medium
unreachable fd00::/48 dev lo proto static metric 2147483647 error -148 pref medium
fd00:192:168:32::/64 via fd00:0:0:64::24 dev br-lan proto static metric 1024 pref medium
fe80::/64 dev ifb4wlan0 proto kernel metric 256 pref medium
fe80::/64 dev eth1 proto kernel metric 256 pref medium
fe80::/64 dev eth0 proto kernel metric 256 pref medium
fe80::/64 dev br-lan proto kernel metric 256 pref medium
fe80::/64 dev wlan0 proto kernel metric 256 pref medium
fe80::/64 dev dsl0 proto kernel metric 256 pref medium
fe80::/64 dev dsl0.7 proto kernel metric 256 pref medium
fe80::/64 dev 6in4-henet proto kernel metric 256 pref medium

If i try to ping an IPv6 host, I get:
traceroute to www.google.com (2a00:1450:4001:821::2004), 30 hops max, 64 byte packets
1traceroute6: sendto: Network unreachable

I tried to tcpdump all interfaces for Proto 41 packets, nothing shows up as if no packets are being sent at all.

The only pingable IPv6 address outside of fd00 and fe80 range is 2001:470:1f0a:af::2 which is my local tunnel endpoint.

My question is: How can I further debug why the kernel thinks the network is unreachable?

2

Did you block IPv4 ICMP Echo-Request recently?

HE has to be able to ping WAN.

1 Like
3
  • Restart the network service and check again.
  • Login to your tunnel broker account and validate the tunnel status.
4

Yes I checked all that /restarted ofc.
The kernel does not even send proto 41 packets anywhere which makes sense cause "Network unreachable" but how can it be unreachable if there is an interface and a default route? there should be at least an outgoing packet.

Also just confirmed pings from outside are being answered and ofc HE knows the correct public ip

5
  • Can you ping ::1?
  • Did you make any firewall changes?
6

This route looks susceptible. Try delete it and see if it works. Restore if it doesn't

Try pinging with source address specified, e.g. ping -I 2xxx:... ipv6.google.com

7

Yes and now... I should mention that eth1 is an lte stick and I also have mwan failover configured...
If I stop the firewall (which kills all iptables rules, also the mwan ones) and then disable eth0 i can suddenly ping via the HE tunnel, but not much more.
Have to investigate this further.
Probably some mwan rule prohibits sending traffic through the henet device. I'm just not understanding why it worked for 3 months, I didn't change anything about the config in that time.

8

Did your WAN fail over?

9

Nope, only strange thing for now is that the henet dev gets an ipv4 address from the eth1 (lte) dhcps range. But I don't think that's a problem as this dev is only intended for v6 traffic.

There is a prerouting hook from mwan in ip6tables. If I'm not wrong this hook intercepts the ipv6 packets and then tries to route them over one of the two failover interfaces which both don't have ipv6 connectivity (thus the ipv6 tunnel).

Now I have to figure out how to tell mwan to not fiddle around with ipv6 rules.

1 Like
10

:confused:

Can you show your config from /etc/config/network for Interface 6in4-henet?

11 
config interface 'henet'
        option proto '6in4'
        option peeraddr '216.66.80.30'
        option ip6addr '2001:470:1f0a:af::2/64'
        option tunnelid 'XXXX'
        option username 'XXX'
        option password 'XXX'
        option ip6prefix '2001:470:7059::/48'
1 Like
12

Up until version 2.0 ipv6 is ignored by mwan3. Ipv6 routing is done without intervention of any mwan3 rule/route. (Source: Adze's post at https://forum.openwrt.org/viewtopic.php?pid=243603#p243603) Starting from version 2.0, you can steer ipv6 traffic if you want.

The link is dead, how exactly do I steer that?

1 Like
13
14

That doesn't seem to have any effect btw, still ip6tables rules added

15

Quick fix for now is to add the following in /etc/mwan3.user
X.X.X.X is tunnel endpoint.
of course this is just a quick hack can be optimized so that it also works when failover device is active.
(see first post for updated problem decription)

ip link set 6in4-henet type sit local $(ip -o address show dev pppoe-wan|cut -f7 -d" ")
ip route del X.X.X.X
ip route add X.X.X.X dev pppoe-wan
16

Replace 'henet' with 'wan6'
/etc/config/firewall
/etc/config/network

Enable wan6 in
/etc/config/mwan3

Add:

 config rule ipv6_update
         option dest_ip 64.62.200.2 #ipv4.tunnelbroker.net
         option use_policy wan_only
 
 config rule ipv6_server
         option dest_ip 216.66.87.102 #tunnel server ip
         option use_policy wan_only
17

are you in china?if so pay attention to gfw to ban google's ipv6 even through he.net's 6in4's tunnel.

18

You should try the latest mwan3 release as it works better with 6in4 interfaces now. Officially it was fixed around 2.8.6, but a minimum of 2.8.8 would be recommended as this version has further IPv6 changes and improvements.

The issue seems to be around busybox ping not working with the interface name, but will work when the /64 on the sit interface is used instead. The mwan3track component would mark the 6in4 interface as down because the ping check would be timing out, but it doesn't fail if the interface source address is passed as the ping target, rather than the interface name. This is a specific quirk with 6in4 interfaces and has been reported before to the core OpenWrt bug tracker. Here's the issue:

This fails:

/bin/ping -I 6in4-wan6 ipv6.google.com ...

This works with the /64 assigned to your side of the tunnel on the sit interface:

/bing/ping -I 2001:470:xxxx:xx:2 ipv6.google.com ...

mwan3track for IPv6 now always uses the IPv6 address and not the interface name to workaround this. Eventually, the idea would be to have the network interface name be used consistently in mwan3track, but right now it is not possible for these reasons. There has been talks around using iputils-ping and making it a dependency, but right now the version in OpenWrt is very old, so it can't be looked at, yet. When the updated version does make it's way into stable, it might be reviewed again.

For now mwan3track looks at the network interface and finds public IPv6 address and then falls back to a /128, and replaces the interface name with whatever IPv6 address it grabs from parsing the network interface info, so this should be fixed. This was one of the major reasons why 6in4 IPv6 was broken horribly, because mwan3track would just keep marking the interface as dead and not route traffic to it, as the ping responses were just timing out, even though the interface was working.

19

FYI, just revisiting this for anyone coming across this issue in the future. It turns out the proper fix is actually a couple of specific kernel patches in netfilter: https://git.openwrt.org/?p=openwrt/openwrt.git;a=commitdiff;h=c9c7b4b3945c01c2aadf3ef5d9a77c8200db80f1

You will need to use OpenWrt 19.07.5 or a recent snapshot build where the kernel patches are included. This does fix issues with interfaces like 6in4 being marked as down all the time.