6in4 and ISP IPv6 with mwan3 load balancing not working

Installing and Using OpenWrt Network and Wireless Configuration
2

mwan3 does not really support IPv6. Well, it does, but it requires a non-standard setup with IPv6 NPT, which is not supported by OpenWrt.

You will need a custom script that sets up NPT for you. Devices in LAN will no longer have public IPv6 addresses, only ULAs, and OpenWrt will translate the prefixes. This is inevitable, as the end hosts have no way to know which source IP to use, so that have to use a temporary one, and the party which knows the correct source IP (OpenWrt) will substitute it.

Here is the script. Save it as /etc/firewall.nat6 and make executable:

#!/bin/sh

source /lib/functions/network.sh

# IPv6 NPT (horrible workaround for the lack of such functionality in OpenWrt)

ip6tables -t nat -F PREROUTING
ip6tables -t nat -F POSTROUTING
ULA=$(uci get network.globals.ula_prefix)
for IFACE in $(uci show mwan3 | sed -n '/=interface/s/^mwan3\.\(.*\)=interface/\1/ p') ; do
  network_get_device DEVICE "$IFACE" || continue
  network_get_prefix6 PREFIX "$IFACE" || continue
  BASE_IFACE=${IFACE%_4}
  BASE_IFACE=${BASE_IFACE%_6}
  network_get_metric METRIC "$BASE_IFACE"
  BITS=${PREFIX##*/}
  if [ "$BITS" -le 48 ] ; then BITS=48 ; fi
  ULA_PART=${ULA%%/*}/$BITS
  FIRST_IP=${PREFIX%%:/*}:1
  echo "Adding a fallback route for $IFACE"
  ADJUSTED_METRIC=$(( METRIC + 20000 ))
  EXISTING_FALLBACK=$(ip -6 route list default from :: dev "$DEVICE" metric "$ADJUSTED_METRIC" )
  if [ -n "$EXISTING_FALLBACK" ] ; then
    echo "$EXISTING_FALLBACK" | sed 's/^/ip route del /' | sh
  fi
  WANTED_ROUTE=$(ip -6 route list default dev "$DEVICE" metric 512 | head -n 1 | sed -e 's/from [0-9a-f:/]* //' -e 's/metric 512 //')
  if [ -n "$WANTED_ROUTE" ] ; then
    ip route add $WANTED_ROUTE dev $DEVICE from :: metric "$ADJUSTED_METRIC"
  fi
  if ip route get "$FIRST_IP" >/dev/null 2>&1 ; then
    echo "Interface $IFACE (on $DEVICE) does not seem to need NPTv6"
    continue
  fi
  echo "Mapping $ULA_PART <-> $PREFIX for $IFACE (on $DEVICE)"
  ip6tables -t nat -A PREROUTING -d $FIRST_IP -j REDIRECT
  ip6tables -t nat -A PREROUTING -d $PREFIX -j NETMAP --to $ULA_PART
  ip6tables -t nat -A POSTROUTING -s $ULA_PART -m conntrack --ctorigdst $PREFIX -j NETMAP --to $PREFIX
  ip6tables -t nat -A POSTROUTING -s $ULA_PART -o $DEVICE -j NETMAP --to $PREFIX
  ip6tables -t nat -A POSTROUTING -s $ULA -o $DEVICE -j MASQUERADE
done

Note: this script requires that each IPv6 WAN has source-based routing, which is false by default for 6in4 tunnels with static IP addresses. Just add this to the tunnel interface configuration:

    option sourcefilter '1'

You would also need the iptables-mod-nat-extra package due to NETMAP.

What this script does is attempt to translate the available public prefixes 1:1 to the beginning of the ULA range, and do a horrible NAT on addresses outside of the mapped area. Also, it helps OpenWrt with selection of the default IPv6 route in situations where source-based routing creates a chicken-and-egg situation ("I cannot choose a source IP, because that would require a valid route to the target, while all routes require that the specified source IP matches").

Then, in /etc/config/firewall, add this:

config include
	option path '/etc/firewall.nat6'

Then, in /etc/config/network, add this to each LAN so that public prefixes are suppressed (because the end devices don't have enough information to select the correct source IP otherwise):

config interface 'lan'
	list ip6class 'local'

In /etc/config/mwan3, create rules and policies as usual (the example below is copy-pasted from my home router, minus the parts beyond the basics):

config globals 'globals'
	option mmx_mask '0x3F00'
	option logging '1'
	option loglevel 'notice'
	list rt_table_lookup '220'

config interface 'wan'
	option enabled '1'
	list track_ip '8.8.4.4'
	list track_ip '8.8.8.8'
	option family 'ipv4'
	option initial_state 'online'
	option track_method 'ping'
	option reliability '1'
	option count '1'
	option size '56'
	option max_ttl '60'
	option timeout '4'
	option interval '10'
	option failure_interval '5'
	option recovery_interval '5'
	option down '5'
	option up '2'
	list flush_conntrack 'connected'
	list flush_conntrack 'disconnected'

config interface 'wan_6'
	option enabled '1'
	list track_ip '2606:4700:4700::1001'
	list track_ip '2606:4700:4700::1111'
	option family 'ipv6'
	option initial_state 'online'
	option track_method 'ping'
	option reliability '1'
	option count '1'
	option size '56'
	option max_ttl '60'
	option timeout '4'
	option interval '10'
	option failure_interval '5'
	option recovery_interval '5'
	option down '5'
	option up '2'
	list flush_conntrack 'connected'
	list flush_conntrack 'disconnected'

config interface 'lte_4'
	option enabled '1'
	option initial_state 'offline'
	option family 'ipv4'
	list track_ip '8.8.8.8'
	list track_ip '8.8.4.4'
	option track_method 'ping'
	option reliability '1'
	option count '1'
	option size '56'
	option max_ttl '60'
	option timeout '4'
	option interval '10'
	option failure_interval '5'
	option recovery_interval '5'
	option down '5'
	option up '2'

config interface 'lte_6'
	option enabled '1'
	option initial_state 'offline'
	option family 'ipv6'
	list track_ip '2606:4700:4700::1001'
	list track_ip '2606:4700:4700::1111'
	option track_method 'ping'
	option reliability '1'
	option count '1'
	option size '56'
	option max_ttl '60'
	option timeout '4'
	option interval '10'
	option failure_interval '5'
	option recovery_interval '5'
	option down '5'
	option up '2'

config rule 'pfx_deleg'
	option family 'ipv6'
	option proto 'all'
        # Substitute with your own ULA prefix
	option dest_ip 'fd7b:7d49:75e3::/48'
	option sticky '0'
	option use_policy 'default'

config rule 'default_rule_v4'
	option dest_ip '0.0.0.0/0'
	option family 'ipv4'
	option proto 'all'
	option sticky '0'
	option use_policy 'failover'

config rule 'default_rule_v6'
	option dest_ip '::/0'
	option family 'ipv6'
	option proto 'all'
	option sticky '0'
	option use_policy 'failover'

config member 'wan_m80'
	option interface 'wan'
	option metric '80'

config member 'wan_6_m80'
	option interface 'wan_6'
	option metric '80'

config member 'lte_4_m90'
	option interface 'lte_4'
	option metric '90'

config member 'lte_6_m90'
	option interface 'lte_6'
	option metric '90'

config policy 'failover'
	option last_resort 'unreachable'
	list use_member 'wan_m80'
	list use_member 'wan_6_m80'
	list use_member 'lte_4_m90'
	list use_member 'lte_6_m90'

config policy 'wan_only'
	list use_member 'wan_m80'
	list use_member 'wan_6_m80'
	option last_resort 'unreachable'

config policy 'lte_only'
	option last_resort 'unreachable'
	list use_member 'lte_4_m90'
	list use_member 'lte_6_m90'

And not, this is not how it is supposed to work from the user experience standpoint. I started a huge thread about doing IPv6 fail-over without NPT, and @justinappler even identified a very good RFC (https://datatracker.ietf.org/doc/rfc8475/) that explains how it should be done. Yet, this is not how mwan3 works. I think they have painted themselves into a corner by committing to support the situation that different uplinks could be chosen based on the destination IP - in which case that RFC is inapplicable.

Or, I have a better idea: just switch off IPv6. It is not ready in the multi-WAN case, and security auditors will love you. Besides, a /64 is not enough for anything except the simplest possible home network.