IPv6 policy routing for VPN and PPPoE


#1
  1. Network Structure:
    (1)Optical Fiber(bridge mode)--->openwrt router-A(pppoe)--->PC/IPHONE/ANDROID device
    (2)Links on openwrt router-A(firmware 18.06.2),
    (a)pppoe: ipv4/ipv6 dual stack, both ipv4 and ipv6 are working well.
    (b)vpn client:private ipv4 address, and public ipv6 address from vpn server. both ipv4 and ipv6 are working well if the default route was set to vpn interface(override the pppoe-wan/wan6).

  2. my goal:
    (1) Specified domain list(about 8000 domains) go through VPN,both on ipv4 and ipv6.
    (2) others go through pppoe-wan and pppoe-wan6.

  3. my approach:
    (1) dnsmasq + ipset: classify the IPs of specified domain list into the ipset "vpnlist4" for ipv4 and "vpnlist6" for ipv6.

ipset -N vpnlist4 iphash
ipset -N vpnlist6 iphash family inet6

(2) iptables/ip6tables to mark the above ipsets.

iptables -t mangle -N fwmark
iptables -t mangle -C PREROUTING -j fwmark || iptables -t mangle -A PREROUTING -j fwmark
iptables -t mangle -C FORWARD -j fwmark || iptables -t mangle -A FORWARD -j fwmark
iptables -t mangle -C OUTPUT -j fwmark || iptables -t mangle -A OUTPUT -j fwmark
iptables -t mangle -A fwmark -m set --match-set vpnlist4 dst -j MARK --set-mark 1
ip6tables -t mangle -N fwmark
ip6tables -t mangle -C PREROUTING -j fwmark || ip6tables -t mangle -A PREROUTING -j fwmark
ip6tables -t mangle -C FORWARD -j fwmark || ip6tables -t mangle -A FORWARD -j fwmark
ip6tables -t mangle -C OUTPUT -j fwmark || ip6tables -t mangle -A OUTPUT -j fwmark
ip6tables -t mangle -A fwmark -m set --match-set vpnlist6 dst -j MARK --set-mark 2

(3)set route table , ip rule etc.

echo "201 vpn4table" >> /etc/iproute2/rt_tables
ip rule add fwmark 1 table 201
ip route add 0.0.0.0/0 via 10.6.1.1 dev vpn table 201
iptables -I FORWARD -i vpn -j ACCEPT
iptables -I FORWARD -o vpn -j ACCEPT
iptables -t nat -I POSTROUTING -o vpn -j MASQUERADE
echo "202 vpn6table" >> /etc/iproute2/rt_tables
ip -6 rule add fwmark 3 table 202
ip -6 route add ::/0 dev vpn table 202
ip6tables -A FORWARD -i wg0 -j ACCEPT
ip6tables -A FORWARD -o wg0 -j ACCEPT
  1. Results:
    (1) ipv4 is working well, all the traffic of specified domain, are go through the vpn.
# ping www.google.com
PING www.google.com (172.217.26.100): 56 data bytes
64 bytes from 172.217.26.100: seq=1 ttl=54 time=178.129 ms
64 bytes from 172.217.26.100: seq=2 ttl=54 time=187.129 ms
64 bytes from 172.217.26.100: seq=3 ttl=54 time=192.024 ms
# traceroute www.google.com
traceroute to www.google.com (172.217.26.100), 30 hops max, 38 byte packets
 1  10.6.1.1 (10.6.1.1)  216.992 ms  212.719 ms  218.490 ms
 2  *  *  *
 3  *  66.42.46.97 (66.42.46.97)  176.827 ms  185.880 ms

**(2) ipv6 isn't working.

# ping6 www.google.com
PING www.google.com (2404:6800:400a:808::2004): 56 data bytes
^C
--- www.google.com ping statistics ---
39 packets transmitted, 0 packets received, 100% packet loss
  1. my analysis and finding:
    (1) By tcpdump sniffer on openwrt router-A, I found that: If I ping6 www.google.com, the packet was go through the vpn interface, but it's came from the ipv6 address which was assigned by the PPPoE, not came from the VPN.
# tcpdump -i vpn ip6 host www.google.com
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vpn, link-type RAW (Raw IP), capture size 262144 bytes
21:26:49.025342 IP6 21xx:xxxx:xxx:aabb::1 > 2404:6800:400a:808::2004: ICMP6, echo request, seq 32, length 64
21:26:50.025439 IP6 21xx:xxxx:xxxx:aabb::1 > 2404:6800:400a:808::2004: ICMP6, echo request, seq 33, length 64
21:26:51.025546 IP6 21xx:xxxx:xxxx:aabb::1 > 2404:6800:400a:808::2004: ICMP6, echo request, seq 34, length 64
^C
7 packets captured
9 packets received by filter
0 packets dropped by kernel

the " 21xx:xxxx:xxxx:aabb::1" was assigned by the PPPoE.
and the "2001:xxxx:xxxx:xxxx:3::2" was assinged by the VPN.

(2) If I set the VPN as the default gateway for all outgoing packet, the ipv6 will be ok.

# ping6 www.google.com
PING www.google.com (2404:6800:4004:80e::2004): 56 data bytes
64 bytes from 2404:6800:4004:80e::2004: seq=0 ttl=56 time=159.679 ms
64 bytes from 2404:6800:4004:80e::2004: seq=1 ttl=56 time=170.058 ms
64 bytes from 2404:6800:4004:80e::2004: seq=2 ttl=56 time=162.111 ms
^C
--- www.google.com ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 159.679/163.949/170.058 ms
# traceroute6 -n www.google.com
traceroute to www.google.com (2404:6800:4004:80e::2004), 30 hops max, 64 byte packets
 1  2001:xxxx:xxxx:xxxx:3::1  173.654 ms  163.451 ms  173.234 ms
 2  *  *  *
 3  *  2001:xxxx:xxxx:1::1  183.164 ms  164.966 ms
 4  *  *^C
  1. Related output:
    (1)
# ip6tables -t mangle -L
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         
fwmark     all      anywhere             anywhere            

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         
fwmark     all      anywhere             anywhere            

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
fwmark     all      anywhere             anywhere            

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         

Chain fwmark (3 references)
target     prot opt source               destination         
MARK       all      anywhere             anywhere             match-set vpnlist6 dst MARK set 0x2
# ip -6 rule list
0:	from all lookup local 
32765:	from all fwmark 0x2 lookup vpn6table 
32766:	from all lookup main
# ip -6 route show table 202
default dev vpn metric 1024 pref medium
  1. HELP ME PLEASE! Thank you very much.

#2

How do you plan to keep ipset up to date, if those domains utilize load balancing?


#3

"up to date" is not the concern.
"it's accessible" is the concern,
because we cann't access www.google.com www.youtube.com www.facebook.com www.twitter.com etc. in my country, we need the vpn to see the another world.


#4

Actually, I understand you.

You should perform more detailed diagnostics:

  • Verify that vpnlist6 matches IPv6-address that you have just pinged.
  • Verify that your setup is logically correct, there are no collisions and all the settings are applied properly:
ip -6 addr show; ip -6 route list table all; ip -6 rule list; ip6tables-save
  • Enable ip6tables logging and verify that your traffic is marked correctly.