Hello openWrt community,
I am the maintainer of the mwan3 package https://github.com/openwrt/packages/tree/master/net/mwan3 .
I want to know the experience if anyone is using mwan3 in a IPv6 only and in a IPv6/IPv4 mixed environment. What are the Problems to get mwan3 working on IPv6. Also I want to get an overview who is using mwan3 and what are the problems and missing feature are.
Thanks
Hi,
I'm trying to get mwan3 working with two dual stack internet connections. IPv4 seems to work but my wanb6 interface is always offline. I get IPv6 addresses from both connection on my client, is this correct? How does the client decide which address to use?
Ping6 for wan6 (this one works):
# ping6 -c 1 -I eth0.1 www.google.com
PING www.google.com (2a00:1450:4001:816::2004): 56 data bytes
ping6: sendto: Permission denied
Ping6 for wanb6 (this one doesn't work):
# ping6 -c 1 -I eth0.2 www.google.com
PING www.google.com (2a00:1450:4001:816::2004): 56 data bytes
64 bytes from 2a00:1450:4001:816::2004: seq=0 ttl=57 time=33.391 ms
--- www.google.com ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 33.391/33.391/33.391 ms
# mwan3 status
Interface status:
interface wan is online and tracking is active
interface wan6 is online and tracking is active
interface wanb is online and tracking is active
interface wanb6 is offline and tracking is active
Current ipv4 policies:
balanced:
wanb (40%)
wan (60%)
wan_only:
wan (100%)
wan_wanb:
wan (100%)
wanb_only:
wanb (100%)
wanb_wan:
wanb (100%)
Current ipv6 policies:
balanced:
wan6 (100%)
wan_only:
wan6 (100%)
wan_wanb:
wan6 (100%)
wanb_only:
unreachable
wanb_wan:
wan6 (100%)
Directly connected ipv4 networks:
224.0.0.0/3
10.0.0.255
127.0.0.0
127.255.255.255
10.0.0.0
10.0.0.1
192.168.179.0/24
192.168.178.184
192.168.178.255
192.168.179.0
192.168.179.255
127.0.0.0/8
10.0.0.0/24
127.0.0.1
192.168.179.25
192.168.178.0
192.168.178.0/24
Directly connected ipv6 networks:
prefix:of:wanb6:address::/64
prefix:of:wan6:address::/56
prefix:of:wanb6:address::/56
fe80::/64
prefix:of:wanb6:pd::/64
fd8b:1382:bd9::/64
pfrefix:of:wan6:pd::/64
prefix:of:wan6:address::/64
Active ipv4 user rules:
2 100 S https tcp -- * * 0.0.0.0/0 0.0.0.0/0 multiport sports 0:65535 multiport dports 443
0 0 - balanced all -- * * 0.0.0.0/0 0.0.0.0/0
Active ipv6 user rules:
0 0 S https tcp * * ::/0 ::/0 multiport sports 0:65535 multiport dports 443
0 0 - balanced all * * ::/0 ::/0
Running openwrt 18.06.2 on x86_64. Didn't touch the default configuration except enabling the interfaces. Both interfaces have an IPv6 address in a /64 and a deligated prefix of /58.
I would appreciate your help!
EDIT: Just installed the latest snapshot and switched wan and wanb. Now wan6 doesn't work, so it seems to be related to the router which connects to the ISP. Those are both FRITZ!boxes (different models) but the relevant configuration should be the same.
Thinking about the issue with multiple wans and ipv6. Based on which IP address is selected by the originator of the traffic, the traffic can only traverse the WAN associated with that prefix. There's not much that mwan3 can do other than trying to get fancy and use NPT6 in some tricky way that would probably cause fragility.
Anyone else have thoughts?
This situation would usually be solved by the ISP (both) accepting packets from both prefixes, but this is generally only an option for (higher end) business contracts - for consumer contracts the ISP won't even consider this request.
Those were my thoughts too.
On ipv4 we have a dedicated network address which will get NATed anyway on the wan interfaces.
On ipv6 we have no dedicated network address so we have to use NAT66 or NPT6 on the wan interfaces.
Since much traffic comes from web access another option is to use a proxy and tell the proxy to use different source IPs for different connections explicitly
I've been trying to use MWAN3 today and I have the same problem, IPv4 works great but IPv6 doesn't and explained above by @dlakelan.
The NAT idea I believe would defeat the purpose, I would actually want to use that mainly for failover.
@feckert,
How do you handle interfaces that have both IPv4 and IPv6? In MWAN3 I can only select IPv4 or IPv6 to use with a single interface. Selecting one of both enables the selected protocol and it works. But is it possible to use IPv4 and IPv6 on a single interface somehow?
In my case I have 3 wireguard interfaces where all 3 have both IPv4 and IPv6 addresses, I'm using failover to switch between them.
I recently managed to make my LTE dongle work, so I gave mwan3 a try. As neither of my ISPs support IPv6 so far, I am using HEnet tunnel for 6in4. However IPv6 doesn't work at all.
I tried to create a set of Interface-Member-Policy-Rule for v6, I tried without, I tried to use main routing table as policy for IPv6. No joy.
mwan3 status
Interface status:
interface wan is online and tracking is active
interface LTE is online and tracking is active
interface HEnet is offline and tracking is active
Current ipv4 policies:
HEnet_only:
default
wan_LTE:
wan (100%)
wan_only:
wan (100%)
Current ipv6 policies:
HEnet_only:
default
wan_LTE:
default
wan_only:
default
Directly connected ipv4 networks:
127.0.0.1
127.255.255.255
172.17.17.0/24
10.0.20.0
XXX.XXX.XXX.XXX
127.0.0.0/8
10.0.20.0/30
10.0.10.15
224.0.0.0/3
172.30.30.0
100.117.168.212
10.0.10.0
10.0.1.0/24
37.205.11.30
127.0.0.0
10.0.20.5
10.0.10.0/28
10.0.20.4/30
172.17.17.255
YYY.YYY.YYY.YYY
216.66.86.122
10.0.20.7
10.0.2.1
100.117.168.208/29
10.0.10.1
10.0.20.4
10.0.3.0/24
10.0.20.2
172.30.30.0/24
100.117.168.208
100.117.168.215
172.30.30.255
172.17.17.1
10.0.20.3
10.0.20.8/30
10.0.2.255
172.30.30.1
10.0.10.16/28
10.0.2.0/24
172.17.17.0
10.0.20.10
10.0.2.0
10.0.20.14
Directly connected ipv6 networks:
fd00:bbbb::/64
2001:470:6e:765::/64
fe80::/64
fd00:bbbb:0:ffff::/64
fe80::/10
fd00:bbbb:0:bbbb::/64
2001:470:::/64
2001:470::/64
fd00:cccc::/64
2001:470:ffff::/64
Active ipv4 user rules:
533 90136 - wan_LTE all -- * * 0.0.0.0/0 0.0.0.0/0
0 0 - HEnet_only all -- * * 0.0.0.0/0 0.0.0.0/0
Active ipv6 user rules:
1967 225K - wan_LTE all * * ::/0 ::/0
0 0 - HEnet_only all * * ::/0 ::/0
Troubleshooting
Software-Version
-------------------------------------------------
OpenWrt - OpenWrt 18.06.4 r7808-ef686b7292
LuCI - git-19.241.48373-38f5f4b
Output of "ip a show"
-------------------------------------------------
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet 10.0.2.1/32 scope global lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1492 qdisc fq_codel state UP group default qlen 1000
link/ether 00:11:22:33:44:57 brd ff:ff:ff:ff:ff:ff
inet6 fe80::215:6dff:fec3:30b7/64 scope link
valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 00:11:22:33:44:57 brd ff:ff:ff:ff:ff:ff
inet6 fe80::15:6dff:fec3:30b7/64 scope link
valid_lft forever preferred_lft forever
4: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN group default qlen 1
link/sit 0.0.0.0 brd 0.0.0.0
5: ifb0: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc hfsc state UNKNOWN group default qlen 32
link/ether 1e:21:42:85:2f:fe brd ff:ff:ff:ff:ff:ff
inet6 fe80::1c21:42ff:fe85:2ffe/64 scope link
valid_lft forever preferred_lft forever
6: ifb1: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN group default qlen 32
link/ether 9e:f4:f2:52:c3:2b brd ff:ff:ff:ff:ff:ff
14: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 100
link/none
inet 10.0.20.2/30 brd 10.0.20.3 scope global tun0
valid_lft forever preferred_lft forever
inet6 fe80::fa3:2163:60ab:f975/64 scope link stable-privacy
valid_lft forever preferred_lft forever
16: wwan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 1000
link/ether 00:1e:10:1f:00:00 brd ff:ff:ff:ff:ff:ff
inet 100.117.168.212/29 brd 100.117.168.215 scope global wwan0
valid_lft forever preferred_lft forever
inet6 fe80::21e:10ff:fe1f:0/64 scope link
valid_lft forever preferred_lft forever
24: eth1.2@eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:11:22:33:44:58 brd ff:ff:ff:ff:ff:ff
inet 172.17.17.1/24 brd 172.17.17.255 scope global eth1.2
valid_lft forever preferred_lft forever
inet6 2001:470:59d2:bbbb:2d4f:671:df3b:58b4/64 scope global noprefixroute
valid_lft forever preferred_lft forever
inet6 2001:470:59d2:bbbb:e483:aadd:fd92:1271/64 scope global deprecated dynamic noprefixroute
valid_lft 5046sec preferred_lft 0sec
inet6 fd00:bbbb::bbbb:9d59:8858:4c4c:f14d/64 scope global noprefixroute
valid_lft forever preferred_lft forever
inet6 fe80::15:6dff:fec3:30b8/64 scope link
valid_lft forever preferred_lft forever
25: eth1.3@eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:11:22:33:44:59 brd ff:ff:ff:ff:ff:ff
inet 172.30.30.1/24 brd 172.30.30.255 scope global eth1.3
valid_lft forever preferred_lft forever
inet6 2001:470:59d2:ffff:b808:e7ec:3a2e:48a3/64 scope global dadfailed tentative noprefixroute
valid_lft forever preferred_lft forever
inet6 2001:470:59d2:ffff:b2d5:fc6f:d0d9:c6c9/64 scope global dadfailed deprecated tentative dynamic noprefixroute
valid_lft 5046sec preferred_lft 0sec
inet6 fd00:bbbb::ffff:1e80:7f1:4b32:3fce/64 scope global dadfailed tentative noprefixroute
valid_lft forever preferred_lft forever
inet6 fe80::15:6dff:fec3:30b9/64 scope link dadfailed tentative
valid_lft forever preferred_lft forever
26: eth1.1@eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 00:11:22:33:44:57 brd ff:ff:ff:ff:ff:ff
inet 10.0.2.1/24 brd 10.0.2.255 scope global eth1.1
valid_lft forever preferred_lft forever
inet6 2001:470:59d2::1/64 scope global dadfailed tentative noprefixroute
valid_lft forever preferred_lft forever
inet6 fd00:bbbb::1/64 scope global dadfailed tentative noprefixroute
valid_lft forever preferred_lft forever
inet6 fe80::15:6dff:fec3:30b7/64 scope link dadfailed tentative
valid_lft forever preferred_lft forever
27: pppoe-wan: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1484 qdisc hfsc state UNKNOWN group default qlen 3
link/ppp
inet YYY.YYY.YYY.YYY peer XXX.XXX.XXX.XXX/32 scope global pppoe-wan
valid_lft forever preferred_lft forever
inet6 fe80::8875:3e1e:6934:5e65/10 scope link
valid_lft forever preferred_lft forever
28: elvetias: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1
link/none
inet 10.0.20.5/30 brd 10.0.20.7 scope global elvetias
valid_lft forever preferred_lft forever
inet6 fd00:cccc::1/64 scope global
valid_lft forever preferred_lft forever
29: roadwarrior: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1
link/none
inet 10.0.10.1/28 brd 10.0.10.15 scope global roadwarrior
valid_lft forever preferred_lft forever
31: 6in4-HEnet@NONE: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1480 qdisc noqueue state UNKNOWN group default qlen 1
link/sit YYY.YYY.YYY.YYY peer 216.66.86.122
inet6 2001:470::2/64 scope global
valid_lft forever preferred_lft forever
inet6 fe80::5f50:d375/64 scope link
valid_lft forever preferred_lft forever
Output of "ip route show"
-------------------------------------------------
default via 10.0.2.1 dev lo
default via XXX.XXX.XXX.XXX dev pppoe-wan proto static metric 10
default via 100.117.168.209 dev wwan0 proto static src 100.117.168.212 metric 40
10.0.1.0/24 via 10.0.20.1 dev tun0 proto zebra metric 20
10.0.2.0/24 dev eth1.1 proto kernel scope link src 10.0.2.1
10.0.3.0/24 via 10.0.10.3 dev roadwarrior proto zebra metric 20
10.0.10.0/28 dev roadwarrior proto kernel scope link src 10.0.10.1
10.0.10.16/28 via 10.0.20.1 dev tun0 proto zebra metric 20
10.0.20.0/30 dev tun0 proto kernel scope link src 10.0.20.2
10.0.20.2 via 10.0.20.1 dev tun0 proto zebra metric 20
10.0.20.4/30 dev elvetias proto kernel scope link src 10.0.20.5
10.0.20.8/30 via 10.0.20.1 dev tun0 proto zebra metric 20
10.0.20.10 via 10.0.20.1 dev tun0 proto zebra metric 20
10.0.20.14 via 10.0.20.1 dev tun0 proto zebra metric 20
37.205.11.30 dev pppoe-wan proto static scope link metric 10
XXX.XXX.XXX.XXX dev pppoe-wan proto kernel scope link src YYY.YYY.YYY.YYY
100.117.168.208/29 dev wwan0 proto static scope link metric 40
172.17.17.0/24 dev eth1.2 proto kernel scope link src 172.17.17.1
172.30.30.0/24 dev eth1.3 proto kernel scope link src 172.30.30.1
216.66.86.122 via XXX.XXX.XXX.XXX dev pppoe-wan proto static
216.66.86.122 via 100.117.168.209 dev wwan0 proto static metric 40
Output of "ip -6 route show"
-------------------------------------------------
default from 2001:470::/64 dev 6in4-HEnet proto static metric 20 pref medium
default from 2001:470::/48 dev 6in4-HEnet proto static metric 20 pref medium
2001:470::/64 dev 6in4-HEnet proto static metric 20 pref medium
2001:470::/64 dev eth1.1 proto static metric 1024 pref medium
2001:470::/64 dev eth1.2 proto static metric 1024 pref medium
2001:470::/64 dev eth1.3 proto static metric 1024 pref medium
unreachable 2001:470::/48 dev lo proto static metric 2147483647 error -148 pref medium
fd00:bbbb::/64 dev eth1.1 proto static metric 1024 pref medium
fd00:bbbb:0:bbbb::/64 dev eth1.2 proto static metric 1024 pref medium
fd00:bbbb:0:ffff::/64 dev eth1.3 proto static metric 1024 pref medium
unreachable fd00:bbbb::/48 dev lo proto static metric 2147483647 error -148 pref medium
fd00:cccc::/64 dev elvetias proto kernel metric 256 pref medium
fe80::/64 dev ifb0 proto kernel metric 256 pref medium
fe80::/64 dev tun0 proto kernel metric 256 pref medium
fe80::/64 dev eth0 proto kernel metric 256 pref medium
fe80::/64 dev eth1 proto kernel metric 256 pref medium
fe80::/64 dev eth1.1 proto kernel metric 256 pref medium
fe80::/64 dev eth1.2 proto kernel metric 256 pref medium
fe80::/64 dev eth1.3 proto kernel metric 256 pref medium
fe80::/64 dev wwan0 proto kernel metric 256 pref medium
fe80::/64 dev 6in4-HEnet proto kernel metric 256 pref medium
fe80::/10 dev pppoe-wan metric 1 pref medium
fe80::/10 dev pppoe-wan proto kernel metric 256 pref medium
Output of "ip rule show"
-------------------------------------------------
0: from all lookup local
1001: from all iif pppoe-wan lookup main
1002: from all iif wwan0 lookup main
2001: from all fwmark 0x100/0x3f00 lookup 1
2002: from all fwmark 0x200/0x3f00 lookup 2
2061: from all fwmark 0x3d00/0x3f00 blackhole
2062: from all fwmark 0x3e00/0x3f00 unreachable
32766: from all lookup main
32767: from all lookup default
Output of "ip -6 rule show"
-------------------------------------------------
0: from all lookup local
1003: from all iif 6in4-HEnet lookup main
2003: from all fwmark 0x300/0x3f00 lookup 3
2061: from all fwmark 0x3d00/0x3f00 blackhole
2062: from all fwmark 0x3e00/0x3f00 unreachable
32766: from all lookup main
4200000000: from 2001:470:59d2::1/64 iif eth1.1 unreachable
4200000000: from 2001:470:59d2:bbbb:2d4f:671:df3b:58b4/64 iif eth1.2 unreachable
4200000000: from 2001:470:59d2:ffff:b808:e7ec:3a2e:48a3/64 iif eth1.3 unreachable
4200000001: from all iif lo failed_policy
4200000014: from all iif tun0 failed_policy
4200000016: from all iif wwan0 failed_policy
4200000016: from all iif wwan0 failed_policy
4200000024: from all iif eth1.2 failed_policy
4200000025: from all iif eth1.3 failed_policy
4200000026: from all iif eth1.1 failed_policy
4200000027: from all iif pppoe-wan failed_policy
4200000028: from all iif elvetias failed_policy
4200000028: from all iif elvetias failed_policy
4200000029: from all iif roadwarrior failed_policy
4200000031: from all iif 6in4-HEnet failed_policy
Output of "ip route list table 1-250"
-------------------------------------------------
Table 1: default via XXX.XXX.XXX.XXX dev pppoe-wan
Table 2: default via 100.117.168.209 dev wwan0
Output of "iptables -L -t mangle -v -n"
-------------------------------------------------
Chain PREROUTING (policy ACCEPT 267K packets, 324M bytes)
pkts bytes target prot opt in out source destination
302K 334M mwan3_hook all -- * * 0.0.0.0/0 0.0.0.0/0
Chain INPUT (policy ACCEPT 10508 packets, 1489K bytes)
pkts bytes target prot opt in out source destination
Chain FORWARD (policy ACCEPT 245K packets, 321M bytes)
pkts bytes target prot opt in out source destination
17M 4855M qos_Default all -- * pppoe-wan 0.0.0.0/0 0.0.0.0/0
181 10732 TCPMSS tcp -- * pppoe-wan 0.0.0.0/0 0.0.0.0/0 tcp flags:0x06/0x02 /* !fw3: Zone wan MTU fixing */ TCPMSS clamp to PMTU
0 0 TCPMSS tcp -- * 6in4-HEnet 0.0.0.0/0 0.0.0.0/0 tcp flags:0x06/0x02 /* !fw3: Zone wan MTU fixing */ TCPMSS clamp to PMTU
0 0 TCPMSS tcp -- * wwan0 0.0.0.0/0 0.0.0.0/0 tcp flags:0x06/0x02 /* !fw3: Zone wan MTU fixing */ TCPMSS clamp to PMTU
Chain OUTPUT (policy ACCEPT 9779 packets, 2166K bytes)
pkts bytes target prot opt in out source destination
28M 3965M qos_Default all -- * pppoe-wan 0.0.0.0/0 0.0.0.0/0
20649 5550K mwan3_hook all -- * * 0.0.0.0/0 0.0.0.0/0
Chain POSTROUTING (policy ACCEPT 255K packets, 323M bytes)
pkts bytes target prot opt in out source destination
Chain mwan3_connected (2 references)
pkts bytes target prot opt in out source destination
254K 328M MARK all -- * * 0.0.0.0/0 0.0.0.0/0 match-set mwan3_connected dst MARK or 0x3f00
Chain mwan3_hook (2 references)
pkts bytes target prot opt in out source destination
322K 340M CONNMARK all -- * * 0.0.0.0/0 0.0.0.0/0 CONNMARK restore mask 0x3f00
25388 2738K mwan3_ifaces_in all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0x3f00
24713 2695K mwan3_connected all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0x3f00
2436 278K mwan3_ifaces_out all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0x3f00
1036 160K mwan3_rules all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0x3f00
322K 340M CONNMARK all -- * * 0.0.0.0/0 0.0.0.0/0 CONNMARK save mask 0x3f00
265K 329M mwan3_connected all -- * * 0.0.0.0/0 0.0.0.0/0 mark match ! 0x3f00/0x3f00
Chain mwan3_iface_in_LTE (1 references)
pkts bytes target prot opt in out source destination
0 0 MARK all -- wwan0 * 0.0.0.0/0 0.0.0.0/0 match-set mwan3_connected src mark match 0x0/0x3f00 /* default */ MARK or 0x3f00
0 0 MARK all -- wwan0 * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0x3f00 /* LTE */ MARK xset 0x200/0x3f00
Chain mwan3_iface_in_wan (1 references)
pkts bytes target prot opt in out source destination
103 13875 MARK all -- pppoe-wan * 0.0.0.0/0 0.0.0.0/0 match-set mwan3_connected src mark match 0x0/0x3f00 /* default */ MARK or 0x3f00
572 29863 MARK all -- pppoe-wan * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0x3f00 /* wan */ MARK xset 0x100/0x3f00
Chain mwan3_iface_out_LTE (1 references)
pkts bytes target prot opt in out source destination
695 58380 MARK all -- * wwan0 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0x3f00 /* LTE */ MARK xset 0x200/0x3f00
Chain mwan3_iface_out_wan (1 references)
pkts bytes target prot opt in out source destination
705 59220 MARK all -- * pppoe-wan 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0x3f00 /* wan */ MARK xset 0x100/0x3f00
Chain mwan3_ifaces_in (1 references)
pkts bytes target prot opt in out source destination
25382 2738K mwan3_iface_in_wan all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0x3f00
24587 2684K mwan3_iface_in_LTE all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0x3f00
Chain mwan3_ifaces_out (1 references)
pkts bytes target prot opt in out source destination
2435 278K mwan3_iface_out_wan all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0x3f00
1726 217K mwan3_iface_out_LTE all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0x3f00
Chain mwan3_policy_HEnet_only (1 references)
pkts bytes target prot opt in out source destination
0 0 MARK all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0x3f00 /* default */ MARK or 0x3f00
Chain mwan3_policy_wan_LTE (1 references)
pkts bytes target prot opt in out source destination
514 87660 MARK all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0x3f00 /* wan 3 3 */ MARK xset 0x100/0x3f00
Chain mwan3_policy_wan_only (0 references)
pkts bytes target prot opt in out source destination
0 0 MARK all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0x3f00 /* wan 3 3 */ MARK xset 0x100/0x3f00
Chain mwan3_rules (1 references)
pkts bytes target prot opt in out source destination
514 87660 mwan3_policy_wan_LTE all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0x3f00 /* default_rule */
0 0 mwan3_policy_HEnet_only all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0x3f00 /* default_rule_v6 */
Chain qos_Default (2 references)
pkts bytes target prot opt in out source destination
45M 8820M CONNMARK all -- * * 0.0.0.0/0 0.0.0.0/0 CONNMARK restore mask 0xf
382K 42M qos_Default_ct all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0xf
30M 3908M MARK udp -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0xf0 length 0:500 MARK xset 0x22/0xff
195K 19M MARK icmp -- * * 0.0.0.0/0 0.0.0.0/0 MARK xset 0x11/0xff
414K 347M MARK tcp -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0xf0 tcp spts:1024:65535 dpts:1024:65535 MARK xset 0x44/0xff
2314K 2382M MARK udp -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0xf0 udp spts:1024:65535 dpts:1024:65535 MARK xset 0x44/0xff
45M 8820M CONNMARK all -- * * 0.0.0.0/0 0.0.0.0/0 CONNMARK save mask 0xff
Chain qos_Default_ct (1 references)
pkts bytes target prot opt in out source destination
78 17714 MARK all -- * * 10.0.2.31 0.0.0.0/0 mark match 0x0/0xf /* spa941 */ MARK xset 0x11/0xff
3351 204K MARK all -- * * 10.0.2.32 0.0.0.0/0 mark match 0x0/0xf /* gigaset-c530 */ MARK xset 0x11/0xff
3182 183K MARK tcp -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0xf tcp multiport ports 22,53 /* ssh, dns */ MARK xset 0x22/0xff
96434 6626K MARK udp -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0xf udp multiport ports 22,53 /* ssh, dns */ MARK xset 0x22/0xff
55588 4907K MARK icmp -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0xf MARK xset 0x22/0xff
139K 8513K MARK tcp -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0xf tcp multiport ports 20,21,25,80,110,143,443,465,993,995 /* ftp, smtp, http(s), imap, pop */ MARK xset 0x33/0xff
84179 22M MARK all -- * * 0.0.0.0/0 0.0.0.0/0 mark match 0x0/0xf MARK xset 0x44/0xff
382K 42M CONNMARK all -- * * 0.0.0.0/0 0.0.0.0/0 CONNMARK save mask 0xff
Output of "ip6tables -L -t mangle -v -n"
-------------------------------------------------
Chain PREROUTING (policy ACCEPT 2351 packets, 269K bytes)
pkts bytes target prot opt in out source destination
3639 416K mwan3_hook all * * ::/0 ::/0
Chain INPUT (policy ACCEPT 21 packets, 3048 bytes)
pkts bytes target prot opt in out source destination
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 qos_Default all * pppoe-wan ::/0 ::/0
0 0 TCPMSS tcp * pppoe-wan ::/0 ::/0 tcp flags:0x06/0x02 /* !fw3: Zone wan MTU fixing */ TCPMSS clamp to PMTU
0 0 TCPMSS tcp * 6in4-HEnet ::/0 ::/0 tcp flags:0x06/0x02 /* !fw3: Zone wan MTU fixing */ TCPMSS clamp to PMTU
0 0 TCPMSS tcp * wwan0 ::/0 ::/0 tcp flags:0x06/0x02 /* !fw3: Zone wan MTU fixing */ TCPMSS clamp to PMTU
Chain OUTPUT (policy ACCEPT 812 packets, 86596 bytes)
pkts bytes target prot opt in out source destination
4974 784K qos_Default all * pppoe-wan ::/0 ::/0
878 95344 mwan3_hook all * * ::/0 ::/0
Chain POSTROUTING (policy ACCEPT 812 packets, 86596 bytes)
pkts bytes target prot opt in out source destination
Chain mwan3_connected (2 references)
pkts bytes target prot opt in out source destination
17 1628 MARK all * * ::/0 ::/0 match-set mwan3_connected dst MARK or 0x3f00
Chain mwan3_hook (2 references)
pkts bytes target prot opt in out source destination
4 224 RETURN icmpv6 * * ::/0 ::/0 ipv6-icmptype 133
26 6864 RETURN icmpv6 * * ::/0 ::/0 ipv6-icmptype 134
4 256 RETURN icmpv6 * * ::/0 ::/0 ipv6-icmptype 135
0 0 RETURN icmpv6 * * ::/0 ::/0 ipv6-icmptype 136
0 0 RETURN icmpv6 * * ::/0 ::/0 ipv6-icmptype 137
4483 504K CONNMARK all * * ::/0 ::/0 CONNMARK restore mask 0x3f00
4390 494K mwan3_ifaces_in all * * ::/0 ::/0 mark match 0x0/0x3f00
4390 494K mwan3_connected all * * ::/0 ::/0 mark match 0x0/0x3f00
4386 494K mwan3_ifaces_out all * * ::/0 ::/0 mark match 0x0/0x3f00
3658 418K mwan3_rules all * * ::/0 ::/0 mark match 0x0/0x3f00
4483 504K CONNMARK all * * ::/0 ::/0 CONNMARK save mask 0x3f00
2090 231K mwan3_connected all * * ::/0 ::/0 mark match ! 0x3f00/0x3f00
Chain mwan3_iface_in_HEnet (1 references)
pkts bytes target prot opt in out source destination
0 0 MARK all 6in4-HEnet * ::/0 ::/0 match-set mwan3_connected_v6 src mark match 0x0/0x3f00 /* default */ MARK or 0x3f00
0 0 MARK all 6in4-HEnet * ::/0 ::/0 mark match 0x0/0x3f00 /* HEnet */ MARK xset 0x300/0x3f00
Chain mwan3_iface_out_HEnet (1 references)
pkts bytes target prot opt in out source destination
728 75712 MARK all * 6in4-HEnet ::/0 ::/0 mark match 0x0/0x3f00 /* HEnet */ MARK xset 0x300/0x3f00
Chain mwan3_ifaces_in (1 references)
pkts bytes target prot opt in out source destination
3093 346K mwan3_iface_in_HEnet all * * ::/0 ::/0 mark match 0x0/0x3f00
Chain mwan3_ifaces_out (1 references)
pkts bytes target prot opt in out source destination
3092 346K mwan3_iface_out_HEnet all * * ::/0 ::/0 mark match 0x0/0x3f00
Chain mwan3_policy_HEnet_only (1 references)
pkts bytes target prot opt in out source destination
0 0 MARK all * * ::/0 ::/0 mark match 0x0/0x3f00 /* default */ MARK or 0x3f00
Chain mwan3_policy_wan_LTE (1 references)
pkts bytes target prot opt in out source destination
2352 269K MARK all * * ::/0 ::/0 mark match 0x0/0x3f00 /* default */ MARK or 0x3f00
Chain mwan3_policy_wan_only (0 references)
pkts bytes target prot opt in out source destination
0 0 MARK all * * ::/0 ::/0 mark match 0x0/0x3f00 /* default */ MARK or 0x3f00
Chain mwan3_rules (1 references)
pkts bytes target prot opt in out source destination
2352 269K mwan3_policy_wan_LTE all * * ::/0 ::/0 mark match 0x0/0x3f00 /* default_rule */
0 0 mwan3_policy_HEnet_only all * * ::/0 ::/0 mark match 0x0/0x3f00 /* default_rule_v6 */
Chain qos_Default (2 references)
pkts bytes target prot opt in out source destination
4974 784K CONNMARK all * * ::/0 ::/0 CONNMARK restore mask 0xf
4950 781K qos_Default_ct all * * ::/0 ::/0 mark match 0x0/0xf
16 2528 MARK udp * * ::/0 ::/0 mark match 0x0/0xf0 length 0:500 MARK xset 0x22/0xff
0 0 MARK icmp * * ::/0 ::/0 MARK xset 0x11/0xff
0 0 MARK tcp * * ::/0 ::/0 mark match 0x0/0xf0 tcp spts:1024:65535 dpts:1024:65535 MARK xset 0x44/0xff
0 0 MARK udp * * ::/0 ::/0 mark match 0x0/0xf0 udp spts:1024:65535 dpts:1024:65535 MARK xset 0x44/0xff
4974 784K CONNMARK all * * ::/0 ::/0 CONNMARK save mask 0xff
Chain qos_Default_ct (1 references)
pkts bytes target prot opt in out source destination
0 0 MARK tcp * * ::/0 ::/0 mark match 0x0/0xf tcp multiport ports 22,53 /* ssh, dns */ MARK xset 0x22/0xff
0 0 MARK udp * * ::/0 ::/0 mark match 0x0/0xf udp multiport ports 22,53 /* ssh, dns */ MARK xset 0x22/0xff
0 0 MARK icmp * * ::/0 ::/0 mark match 0x0/0xf MARK xset 0x22/0xff
0 0 MARK tcp * * ::/0 ::/0 mark match 0x0/0xf tcp multiport ports 20,21,25,80,110,143,443,465,993,995 /* ftp, smtp, http(s), imap, pop */ MARK xset 0x33/0xff
4950 781K MARK all * * ::/0 ::/0 mark match 0x0/0xf MARK xset 0x44/0xff
4950 781K CONNMARK all * * ::/0 ::/0 CONNMARK save mask 0xff
Did you set 0.0.0.0/0 as destination for the ipv4 rule and :: for the ipv6 rule? but its probably best to leave ipv6 out of mwan3 all together and use the main routing table for ipv6
Yes, I tried it like this as well. I believe this is the example I posted. For some reason it uses IPv4 rules for IPv6 traffic. The sequence of the rules didn't seem to make any difference.
I tried to leave IPv6 out of it, but it is still broken.
@feckert I also have a similar problem as @trendy but it may be unrelated to mwan3. I found out that when 6rd interface is up, I can manually reload firewall to get things working. (at least sometimes)
I also had ipv4 rule in ipv6 section like @trendy ...
Also the 6rd interface is created automatically so I am not able to assign a metric to it..
Also I had a strange problem. I setup the rule but way later in logs I saw this:
Wed Nov 13 01:53:41 2019 user.warn mwan3[25498]: Rule default_rule_ipv6 exceeds max of 15 chars. Not setting rule
This is just bad 
Perhaps it is bad that mwan3 is trying to setup ipv6 things right now. It seems broken. Perhaps ipv6 functionality should be developed separately or at least there should be a way to disable it if one desires.
@feckert Can an option to disable mwan3 altogether be added? So system works like it did not exist, perhaps at least after a reboot? This may be useful for testing if mwan3 is causing troubles perhaps?
Without mwan3 it was working fine though, without the route even for the locally generated traffic.
It was when I enabled mwan3 that problems started.
Right now I have disabled mwan3 and I am trying to move to IPv6 from my ISP (more on that on a new thread), so I cannot test it anymore.
Stop mwan3
/etc/init.d/mwan3 stop
Disable mwan3
/etc/init.d/mwan3 disable
@feckert I was stupid to not think about it. But I think there could be a enable/disable checkbox in luci interface. Perhaps not a very important feature request anyway
Pls, separate IPv4 and IPv6 related code.
And obey to
make menue config ; Global build settings ; Enable IPv6 support in packages
Unticking this option results in lot of error messages during startup of mwan3. Very annoying.
My 3g-wwan does not support IPv6. But to avoid all the error messages I build custom image incl. IPv6 just because of this.
Thanks for feedback.
That is not so easy. This was not thought of during the implementation. I have to see how elaborate this is.
Just adding my trials with mwan3 and IPv6. My setup is a bit unusual, but I've got two WAN interfaces.
WANB is on it's own VLAN, essentially I've turned one of the LAN ports into a WAN. WAN6 and WANB are attached to the WAN filewall zone.
I can't get mwan3 and 6in4 to co-exist nicely, no matter what I do, mwan3 states the interface is down after about 30 seconds every time of starting it. I'm assuming protocol 41 is being interfered with somewhere. Even setting a rule in mwan3 itself to make sure traffic to the HE tunnel endpoint goes over the right WAN doesn't work, judging by the iptables output, the rule is never hit, so potentially protocol 41 never makes it.
I have also noticed though that for wanb6, I get strange issues with ping6, traceroute6, curl etc with some IPv6 hosts (not all). e.g.
root@linksys-wrt3200acm:~# ping6 -I eth0.3 ifconfig.co
connect: Permission denied
eth0.3 is the network interface created from the VLAN.
This doesn't happen on the 6in4-wan6 interface and other IPv6 sites like ipv6.google.com are OK, so not quite sure why that's the case.
I have had to give up trying to control IPv6 through mwan3 and simply fallback to the main routing table. Rules using IPv4 with WAN and WANB work great though.
Overall, IPv4 works great, IPv6 is a bit hit and miss. I think if I had native IPv6 on WAN6, it would possibly be less of a problem, it seems 6in4 is particularly problematic with the mwan3 routing.
Thank you however, for the work that has been done with mwan3 up to this point.