[Solved] Configuration of VPN Policy Routing (doesn’t work)

Dominik-1980 · January 2, 2021, 2:22pm

Hello.

I changed from vpnbypass to vpn-policy-routing.
It doesn’t work at the moment.
By default, there was only the VPN Interface available.
So I defined the supported interfaces manually over LUCI GUI.
tun0 and br-lan.
The package recognized this change and VyprVPN and LAN are now available in routing policies.
For testing purposes I tried to route the domain wieistmeineip.de to the LAN interface.
But it doesn’t work.
With vpnbypass I had to wait about one hour and then my changes took effect.
With vpn-policy-routing it,doesn’t work even after waiting for a couple of hours...


root@OpenWrt:~# iptables-save -c -t mangle
# Generated by iptables-save v1.8.3 on Sat Jan  2 15:22:03 2021
*mangle
:PREROUTING ACCEPT [309143:287931467]
:INPUT ACCEPT [129430:142739570]
:FORWARD ACCEPT [166603:136153409]
:OUTPUT ACCEPT [64378:22533462]
:POSTROUTING ACCEPT [230975:158686175]
:VPR_FORWARD - [0:0]
:VPR_INPUT - [0:0]
:VPR_OUTPUT - [0:0]
:VPR_PREROUTING - [0:0]
[309259:287996485] -A PREROUTING -m mark --mark 0x0/0xff0000 -j VPR_PREROUTING
[129482:142773624] -A INPUT -m mark --mark 0x0/0xff0000 -j VPR_INPUT
[499:32691] -A FORWARD -o tun0 -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment --comment "!fw3: Zone VyprVPN MTU fixing" -j TCPMSS --clamp-mss-to-pmtu
[490:29372] -A FORWARD -i tun0 -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment --comment "!fw3: Zone VyprVPN MTU fixing" -j TCPMSS --clamp-mss-to-pmtu
[166662:136177965] -A FORWARD -m mark --mark 0x0/0xff0000 -j VPR_FORWARD
[64428:22541029] -A OUTPUT -m mark --mark 0x0/0xff0000 -j VPR_OUTPUT
[0:0] -A VPR_FORWARD -m set --match-set VyprVPN dst -j MARK --set-xmark 0x20000/0xff0000
[0:0] -A VPR_FORWARD -m set --match-set lan dst -j MARK --set-xmark 0x10000/0xff0000
[0:0] -A VPR_INPUT -m set --match-set VyprVPN dst -j MARK --set-xmark 0x20000/0xff0000
[0:0] -A VPR_INPUT -m set --match-set lan dst -j MARK --set-xmark 0x10000/0xff0000
[0:0] -A VPR_OUTPUT -m set --match-set VyprVPN dst -j MARK --set-xmark 0x20000/0xff0000
[0:0] -A VPR_OUTPUT -m set --match-set lan dst -j MARK --set-xmark 0x10000/0xff0000
[0:0] -A VPR_PREROUTING -m set --match-set VyprVPN dst -j MARK --set-xmark 0x20000/0xff0000
[0:0] -A VPR_PREROUTING -m set --match-set lan dst -j MARK --set-xmark 0x10000/0xff0000
COMMIT
# Completed on Sat Jan  2 15:22:03 2021
root@OpenWrt:~#

trendy · January 2, 2021, 4:18pm

Known-issues

When using the dnsmasq.ipset option, please make sure to flush the DNS cache of the local devices, otherwise domain policies may not work until you do. If you’re not sure how to flush the DNS cache (or if the device/OS doesn’t offer an option to flush its DNS cache), reboot your local devices when starting to use the service and/or when connecting data-capable device to your WiFi.

Because the ipset command only adds a first resolved IP address of the domain on add, if the domain name is encountered as the dest_addr option of the policy (with no other fields set for the policy), it will be attempted to be added as dnsmasq.ipset (if resolver_ipset is set to dnsmasq.ipset), otherwise, the domain name will be resolved when the service starts up and the resolved IP addresses added as either ipset (if enabled) or iptables rules. Resolving a number of domains on start is a time consuming operation, that’s why the use of dnsmasq.ipset value for resolver_ipset options is a preferred scenario.

Also follow the instructions for getting help.

Dominik-1980 · January 2, 2021, 7:24pm

It doesn't work...

0:0 everywhere

root@OpenWrt:~# iptables-save -c -t mangle
# Generated by iptables-save v1.8.3 on Sat Jan  2 20:23:58 2021
*mangle
:PREROUTING ACCEPT [64501:44992196]
:INPUT ACCEPT [27482:20638411]
:FORWARD ACCEPT [30558:20044239]
:OUTPUT ACCEPT [21291:6405613]
:POSTROUTING ACCEPT [51823:26448788]
:VPR_FORWARD - [0:0]
:VPR_INPUT - [0:0]
:VPR_OUTPUT - [0:0]
:VPR_PREROUTING - [0:0]
[64504:44992904] -A PREROUTING -m mark --mark 0x0/0xff0000 -j VPR_PREROUTING
[27483:20638463] -A INPUT -m mark --mark 0x0/0xff0000 -j VPR_INPUT
[30558:20044239] -A FORWARD -m mark --mark 0x0/0xff0000 -j VPR_FORWARD
[263:16840] -A FORWARD -o tun0 -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment --comment "!fw3: Zone VyprVPN MTU fixing" -j TCPMSS --clamp-mss-to-pmtu
[267:15984] -A FORWARD -i tun0 -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment --comment "!fw3: Zone VyprVPN MTU fixing" -j TCPMSS --clamp-mss-to-pmtu
[21292:6405665] -A OUTPUT -m mark --mark 0x0/0xff0000 -j VPR_OUTPUT
[0:0] -A VPR_FORWARD -m set --match-set VyprVPN dst -j MARK --set-xmark 0x20000/0xff0000
[0:0] -A VPR_FORWARD -m set --match-set lan dst -j MARK --set-xmark 0x10000/0xff0000
[0:0] -A VPR_INPUT -m set --match-set VyprVPN dst -j MARK --set-xmark 0x20000/0xff0000
[0:0] -A VPR_INPUT -m set --match-set lan dst -j MARK --set-xmark 0x10000/0xff0000
[0:0] -A VPR_OUTPUT -m set --match-set VyprVPN dst -j MARK --set-xmark 0x20000/0xff0000
[0:0] -A VPR_OUTPUT -m set --match-set lan dst -j MARK --set-xmark 0x10000/0xff0000
[0:0] -A VPR_PREROUTING -m set --match-set VyprVPN dst -j MARK --set-xmark 0x20000/0xff0000
[0:0] -A VPR_PREROUTING -m set --match-set lan dst -j MARK --set-xmark 0x10000/0xff0000
COMMIT
# Completed on Sat Jan  2 20:23:58 2021
root@OpenWrt:~#

trendy · January 2, 2021, 7:53pm

No need to post again the same.
You didn't post the essential:

Dominik-1980 · January 2, 2021, 9:41pm

/etc/config/dhcp

config dnsmasq
        option domainneeded '1'
        option boguspriv '1'
        option filterwin2k '0'
        option localise_queries '1'
        option rebind_protection '1'
        option rebind_localhost '1'
        option local '/lan/'
        option domain 'lan'
        option expandhosts '1'
        option nonegcache '0'
        option authoritative '1'
        option readethers '1'
        option leasefile '/tmp/dhcp.leases'
        option resolvfile '/tmp/resolv.conf.auto'
        option nonwildcard '1'
        option localservice '1'
        option confdir '/tmp/dnsmasq.d'
        list server '127.0.0.1#5453'
        option noresolv '1'
        option dnssec '1'
        option dnsseccheckunsigned '1'
        list ipset '/zh2-9-hls7enc-live.zahs.tv/wieistmeineip.de/vpnbypass'

config dhcp 'lan'
        option interface 'lan'
        option dhcpv6 'server'
        option ra 'server'
        option ra_management '1'
        option ignore '1'

config dhcp 'wan'
        option interface 'wan'
        option ignore '1'

config odhcpd 'odhcpd'
        option maindhcp '0'
        option leasefile '/tmp/hosts/odhcpd'
        option leasetrigger '/usr/sbin/odhcpd-update'
        option loglevel '4'

/etc/config/firewall

config defaults
	option syn_flood '1'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'

config zone
	option name 'lan'
	list network 'lan'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'

config zone
	option name 'wan'
	list network 'wan'
	list network 'wan6'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'

config forwarding
	option src 'lan'
	option dest 'wan'

config rule
	option name 'Allow-DHCP-Renew'
	option src 'wan'
	option proto 'udp'
	option dest_port '68'
	option target 'ACCEPT'
	option family 'ipv4'

config rule
	option name 'Allow-Ping'
	option src 'wan'
	option proto 'icmp'
	option icmp_type 'echo-request'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-IGMP'
	option src 'wan'
	option proto 'igmp'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-DHCPv6'
	option src 'wan'
	option proto 'udp'
	option src_ip 'fc00::/6'
	option dest_ip 'fc00::/6'
	option dest_port '546'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-MLD'
	option src 'wan'
	option proto 'icmp'
	option src_ip 'fe80::/10'
	list icmp_type '130/0'
	list icmp_type '131/0'
	list icmp_type '132/0'
	list icmp_type '143/0'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Input'
	option src 'wan'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	list icmp_type 'router-solicitation'
	list icmp_type 'neighbour-solicitation'
	list icmp_type 'router-advertisement'
	list icmp_type 'neighbour-advertisement'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Forward'
	option src 'wan'
	option dest '*'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-IPSec-ESP'
	option src 'wan'
	option dest 'lan'
	option proto 'esp'
	option target 'ACCEPT'

config rule
	option name 'Allow-ISAKMP'
	option src 'wan'
	option dest 'lan'
	option dest_port '500'
	option proto 'udp'
	option target 'ACCEPT'

config include
	option path '/etc/firewall.user'

config zone
	option network 'VyprVPN'
	option name 'VyprVPN'
	option mtu_fix '1'
	option input 'REJECT'
	option forward 'REJECT'
	option masq '1'
	option output 'ACCEPT'

config forwarding
	option dest 'VyprVPN'
	option src 'lan'

/etc/config/network

config interface 'loopback'
	option ifname 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fd7a:c322:7f86::/48'

config interface 'lan'
	option type 'bridge'
	option ifname 'eth0'
	option proto 'static'
	option ipaddr '192.168.1.2'
	option netmask '255.255.255.0'
	option ip6assign '60'
	option gateway '192.168.1.1'
	list dns '192.168.1.2'

config interface 'VyprVPN'
	option ifname 'tun0'
	option proto 'none'

/etc/config/vpn-policy-routing

config vpn-policy-routing 'config'
	option verbosity '2'
	option strict_enforcement '1'
	option ipv6_enabled '0'
	list ignored_interface 'vpnserver wgserver'
	option boot_timeout '30'
	option iprule_enabled '0'
	option webui_sorting '1'
	list webui_supported_protocol 'tcp'
	list webui_supported_protocol 'udp'
	list webui_supported_protocol 'tcp udp'
	list webui_supported_protocol 'icmp'
	list webui_supported_protocol 'all'
	list supported_interface 'br-lan'
	list supported_interface 'tun0'
	option enabled '1'
	option dest_ipset 'dnsmasq.ipset'
	option iptables_rule_option 'insert'
	option webui_enable_column '1'
	option webui_protocol_column '1'
	option webui_chain_column '1'
	option src_ipset '0'

config include
	option path '/etc/vpn-policy-routing.netflix.user'
	option enabled '0'

config include
	option path '/etc/vpn-policy-routing.aws.user'
	option enabled '0'

config policy
	option name 'Test'
	option dest_addr 'wieistmeineip.de'
	option interface 'lan'
	option proto 'all'

running command /etc/init.d/vpn-policy-routing support

vpn-policy-routing 0.2.1-13 running on OpenWrt 19.07.5. WAN (IPv4): lan/dev/192.168.1.1.
============================================================
Dnsmasq version 2.80  Copyright (c) 2000-2018 Simon Kelley
Compile time options: IPv6 GNU-getopt no-DBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP conntrack ipset auth DNSSEC no-ID loop-detect inotify dumpfile
============================================================
Routes/IP Rules
default         10.2.23.1       128.0.0.0       UG    0      0        0 tun0
default         192.168.1.1     0.0.0.0         UG    0      0        0 br-lan
IPv4 Table 201: default via 192.168.1.1 dev br-lan
IPv4 Table 201 Rules:
32755:	from all fwmark 0x10000/0xff0000 lookup 201
IPv4 Table 202: default via 10.2.23.21 dev tun0
IPv4 Table 202 Rules:
32754:	from all fwmark 0x20000/0xff0000 lookup 202
============================================================
IP Tables PREROUTING
-N VPR_PREROUTING
-A VPR_PREROUTING -m set --match-set VyprVPN dst -c 0 0 -j MARK --set-xmark 0x20000/0xff0000
-A VPR_PREROUTING -m set --match-set lan dst -c 0 0 -j MARK --set-xmark 0x10000/0xff0000
============================================================
IP Tables FORWARD
-N VPR_FORWARD
-A VPR_FORWARD -m set --match-set VyprVPN dst -c 0 0 -j MARK --set-xmark 0x20000/0xff0000
-A VPR_FORWARD -m set --match-set lan dst -c 0 0 -j MARK --set-xmark 0x10000/0xff0000
============================================================
IP Tables INPUT
-N VPR_INPUT
-A VPR_INPUT -m set --match-set VyprVPN dst -c 0 0 -j MARK --set-xmark 0x20000/0xff0000
-A VPR_INPUT -m set --match-set lan dst -c 0 0 -j MARK --set-xmark 0x10000/0xff0000
============================================================
IP Tables OUTPUT
-N VPR_OUTPUT
-A VPR_OUTPUT -m set --match-set VyprVPN dst -c 0 0 -j MARK --set-xmark 0x20000/0xff0000
-A VPR_OUTPUT -m set --match-set lan dst -c 0 0 -j MARK --set-xmark 0x10000/0xff0000
============================================================
Current ipsets
create lan hash:net family inet hashsize 1024 maxelem 65536 comment
create VyprVPN hash:net family inet hashsize 1024 maxelem 65536 comment
create lan_ip hash:net family inet hashsize 1024 maxelem 65536 comment
create lan_mac hash:mac hashsize 1024 maxelem 65536 comment
create VyprVPN_ip hash:net family inet hashsize 1024 maxelem 65536 comment
create VyprVPN_mac hash:mac hashsize 1024 maxelem 65536 comment
============================================================
DNSMASQ ipsets
ipset=/wieistmeineip.de/lan # Test
============================================================
Your support details have been logged to '/var/vpn-policy-routing-support'. [✓]

running command /etc/init.d/vpn-policy-routing reload

Creating table 'lan/br-lan/192.168.1.1' [✓]
Creating table 'VyprVPN/tun0/10.2.23.21' [✓]
Routing 'Test' via lan [✓]
vpn-policy-routing 0.2.1-13 started with gateways:
lan/br-lan/192.168.1.1
VyprVPN/tun0/10.2.23.21 [✓]
vpn-policy-routing 0.2.1-13 monitoring interfaces: lan VyprVPN .

I don't know if it matters, but I use stubby with two custom DNS Servers and DNS over TLS with port-forwarding from port 53 to port 5453...
For installation I did this...

uci add_list dhcp.@dnsmasq[-1].server='127.0.0.1#5453'
uci set dhcp.@dnsmasq[-1].noresolv=1
uci commit && reload_config

uci set network.wan.peerdns='0'
uci set network.wan.dns='127.0.0.1'
uci set network.wan6.peerdns='0'
uci set network.wan6.dns='0::1'
uci commit && reload_config

and here is my stubby config:

config stubby 'global'
       option manual '0'
       option trigger 'wan'
       # option triggerdelay '2'
       list dns_transport 'GETDNS_TRANSPORT_TLS'
       option tls_authentication '1'
       option tls_query_padding_blocksize '128'
       # option tls_connection_retries '2'
       # option tls_backoff_time '3600'
       # option timeout '5000'
       # option dnssec_return_status '0'
       option appdata_dir '/var/lib/stubby'
       # option trust_anchors_backoff_time 2500
       # option dnssec_trust_anchors '/var/lib/stubby/getdns-root.key'
       option edns_client_subnet_private '1'
       option idle_timeout '10000'
       option round_robin_upstreams '1'
       list listen_address '127.0.0.1@5453'
       list listen_address '0::1@5453'
       # option log_level '7'
       # option command_line_arguments ''
       # option tls_cipher_list 'EECDH+AESGCM:EECDH+CHACHA20'
       # option tls_ciphersuites 'TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256'
       option tls_min_version '1.2'
       # option tls_max_version '1.3'

# Upstream resolvers are specified using 'resolver' sections.

config resolver  
        option address '80.241.218.68'  
        option tls_auth_name 'fdns1.dismail.de'  
        list spki 'sha256/MMi3E2HZr5A5GL+badqe3tzEPCB00+OmApZqJakbqUU='

config resolver         
        option address '46.182.19.48' 
        option tls_auth_name 'dns2.digitalcourage.de'    
        list spki 'sha256/v7rm6OtQQD3x/wbsdHDZjiDg+utMZvnoX3jq3Vi8tGU='

trendy · January 2, 2021, 10:07pm

I just gave it a try and it works fine.
A couple of things to point out.
The actual site that shows your IP is www.wieistmeineip.de and has different IP than wieistmeineip.de
It also has IPv6, which by default has priority over IPv4, in case your systems got it.

Dominik-1980 · January 2, 2021, 10:13pm

No, I just have IPv4.
I have edited my previous posting, because I'm using stubby.
Please have a look.

IPv6 is disabled in VPN Policy Routing config.

trendy · January 2, 2021, 10:29pm

From one hand you are using the dnsmasq.ipset (I hope you installed dnsmasq-full for that to work)

and on the other hand you have noresolv=1 in dhcp which makes the local system not to use dnsmasq. And I don't see the localuse=1

These are not used with the configuration you have above.

Is the device able to resolve anything?

Dominik-1980 · January 2, 2021, 10:33pm

Everything works. Internet access and openVPN...
Should I set noresolve=0?
And yes, I have installed dnsmasq-full.

Dominik-1980 · January 2, 2021, 10:48pm

The only thing that changes if I set noresolv=0 is that these two sections:

using local addresses only for ...
using standard name servers for ...

shows up twice in the sys log

Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: started, version 2.80 cachesize 150
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: DNS service limited to local subnets
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: compile time options: IPv6 GNU-getopt no-DBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP conntrack ipset auth DNSSEC no-ID loop-detect inotify dumpfile
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: DNSSEC validation enabled
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: configured with trust anchor for <root> keytag 20326
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: configured with trust anchor for <root> keytag 19036
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using local addresses only for domain test
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using local addresses only for domain onion
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using local addresses only for domain localhost
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using local addresses only for domain local
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using local addresses only for domain invalid
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using local addresses only for domain bind
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using standard nameservers for domain clients4.google.com
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using standard nameservers for domain clients2.google.com
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using standard nameservers for domain video-stats.l.google.com
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using standard nameservers for domain s.youtube.com
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using standard nameservers for domain s.mzstatic.com
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using standard nameservers for domain itunes.apple.com
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using standard nameservers for domain appleid.apple.com
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using standard nameservers for domain com.apple.systempreferences
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using nameserver 127.0.0.1#5453
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using local addresses only for domain lan
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: reading /tmp/resolv.conf.auto
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using local addresses only for domain test
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using local addresses only for domain onion
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using local addresses only for domain localhost
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using local addresses only for domain local
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using local addresses only for domain invalid
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using local addresses only for domain bind
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using standard nameservers for domain clients4.google.com
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using standard nameservers for domain clients2.google.com
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using standard nameservers for domain video-stats.l.google.com
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using standard nameservers for domain s.youtube.com
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using standard nameservers for domain s.mzstatic.com
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using standard nameservers for domain itunes.apple.com
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using standard nameservers for domain appleid.apple.com
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using standard nameservers for domain com.apple.systempreferences
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using nameserver 127.0.0.1#5453
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: using local addresses only for domain lan
Sat Jan  2 23:41:15 2021 daemon.warn dnsmasq[30929]: ignoring nameserver 192.168.1.2 - local interface
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: read /etc/hosts - 4 addresses
Sat Jan  2 23:41:15 2021 daemon.info dnsmasq[30929]: read /tmp/hosts/odhcpd - 3 addresses
Sat Jan  2 23:41:15 2021 daemon.err dnsmasq[30929]: failed to load names from /tmp/hosts/dhcp.cfg01411c: Permission denied
Sat Jan  2 23:41:22 2021 daemon.err modprobe: xt_set is already loaded
Sat Jan  2 23:41:22 2021 daemon.err modprobe: ip_set is already loaded
Sat Jan  2 23:41:22 2021 daemon.err modprobe: ip_set_hash_ip is already loaded
Sat Jan  2 23:41:23 2021 user.notice vpn-policy-routing [30938]: Creating table 'lan/br-lan/192.168.1.1' [✓]
Sat Jan  2 23:41:23 2021 user.notice vpn-policy-routing [30938]: Creating table 'VyprVPN/tun0/10.2.23.45' [✓]
Sat Jan  2 23:41:23 2021 user.notice vpn-policy-routing [30938]: Routing 'Test' via lan [✓]
Sat Jan  2 23:41:23 2021 user.notice vpn-policy-routing [30938]: service started with gateways: lan/br-lan/192.168.1.1 VyprVPN/tun0/10.2.23.45 [✓]
Sat Jan  2 23:41:23 2021 user.notice vpn-policy-routing [30938]: service monitoring interfaces: lan VyprVPN .

Dominik-1980 · January 2, 2021, 10:58pm

with resolve anything you mean this?

trendy · January 2, 2021, 10:59pm

The problem is that although you have the ipset declared

there is no iptables rule to match that.
I added these couple of rules:

config policy
        option name 'meineip2'
        option dest_addr 'www.wieistmeineip.de'
        option chain 'OUTPUT'
        option interface 'proton'

config policy
        option dest_addr 'wieistmeineip.de'
        option interface 'proton'
        option chain 'OUTPUT'
        option name 'meineip'

and I got

-A VPR_OUTPUT -d 52.29.79.39/32 -m comment --comment meineip_wieistmeineip_de -c 0 0 -j MARK --set-xmark 0x30000/0xff0000
-A VPR_OUTPUT -d 35.157.134.147/32 -m comment --comment meineip2_www_wieistmeineip_de -c 0 0 -j MARK --set-xmark 0x30000/0xff0000
-A VPR_OUTPUT -d 52.28.125.65/32 -m comment --comment meineip2_www_wieistmeineip_de -c 0 0 -j MARK --set-xmark 0x30000/0xff0000
-A VPR_OUTPUT -d 52.29.58.77/32 -m comment --comment meineip2_www_wieistmeineip_de -c 0 0 -j MARK --set-xmark 0x30000/0xff0000

which is missing from your iptables. (ignore the OUTPUT)
Do an ipset list to verify if the ipset has any entry.

Dominik-1980 · January 2, 2021, 11:11pm

Adblock works perfectly, too.
This is also using dnsmasq.
Strange... the only thing that doesn't work is the vpn policy routing package...

Dominik-1980 · January 2, 2021, 11:18pm

root@OpenWrt:~# ipset list
Name: lan
Type: hash:net
Revision: 6
Header: family inet hashsize 1024 maxelem 65536 comment
Size in memory: 344
References: 4
Number of entries: 0
Members:

Name: VyprVPN
Type: hash:net
Revision: 6
Header: family inet hashsize 1024 maxelem 65536 comment
Size in memory: 344
References: 4
Number of entries: 0
Members:
root@OpenWrt:~#

Hmmm... seems to have no entry at all...

Can that be significant?

Sun Jan  3 00:23:57 2021 daemon.err modprobe: xt_set is already loaded
Sun Jan  3 00:23:57 2021 daemon.err modprobe: ip_set is already loaded
Sun Jan  3 00:23:58 2021 daemon.err modprobe: ip_set_hash_ip is already loaded

It shows up every time I restart the package...

Sun Jan  3 00:29:14 2021 daemon.err modprobe: xt_set is already loaded
Sun Jan  3 00:29:14 2021 daemon.err modprobe: ip_set is already loaded
Sun Jan  3 00:29:14 2021 daemon.err modprobe: ip_set_hash_ip is already loaded
Sun Jan  3 00:29:14 2021 user.notice vpn-policy-routing [8916]: Creating table 'lan/br-lan/192.168.1.1' [✓]
Sun Jan  3 00:29:14 2021 user.notice vpn-policy-routing [8916]: Creating table 'VyprVPN/tun0/10.2.23.46' [✓]
Sun Jan  3 00:29:15 2021 user.notice vpn-policy-routing [8916]: Routing 'meineip2' via lan [✓]
Sun Jan  3 00:29:16 2021 user.notice vpn-policy-routing [8916]: Routing 'meineip' via lan [✓]

trendy · January 3, 2021, 1:37am

You can ignore the modprobe errors. It's not good that there are no entries in the ipset. Did you restart the services of vpn-pbr, dnsmasq, and flushed the dns caches?

I tried to adapt your configuration in my router and it is still working.

[0:0] -A VPR_OUTPUT -d 52.29.79.39/32 -m comment --comment meineip_wieistmeineip_de -j MARK --set-xmark 0x30000/0xff0000
[0:0] -A VPR_OUTPUT -d 52.29.58.77/32 -m comment --comment meineip2_www_wieistmeineip_de -j MARK --set-xmark 0x30000/0xff0000
[59:5538] -A VPR_OUTPUT -d 35.157.134.147/32 -m comment --comment meineip2_www_wieistmeineip_de -j MARK --set-xmark 0x30000/0xff0000
[0:0] -A VPR_OUTPUT -d 52.28.125.65/32 -m comment --comment meineip2_www_wieistmeineip_de -j MARK --set-xmark 0x30000/0xff0000

My configuration for your reference:

root@magiatiko ~ > uci export vpn-policy-routing
package vpn-policy-routing

config policy
        option name 'meineip2'
        option dest_addr 'www.wieistmeineip.de'
        option interface 'proton'

config policy
        option dest_addr 'wieistmeineip.de'
        option interface 'proton'
        option name 'meineip'

config vpn-policy-routing 'config'
        option ipv6_enabled '0'
        list supported_interface ''
        option boot_timeout '30'
        option webui_sorting '1'
        list webui_supported_protocol 'tcp'
        list webui_supported_protocol 'udp'
        list webui_supported_protocol 'tcp udp'
        list webui_supported_protocol 'icmp'
        list webui_supported_protocol 'all'
        option webui_enable_column '1'
        option webui_protocol_column '1'
        option webui_chain_column '1'
        list ignored_interface 'vpnserver wgserver'
        option iprule_enabled '0'
        option verbosity '1'
        option strict_enforcement '1'
        option enabled '1'
        option iptables_rule_option 'append'
        option dest_ipset 'ipset'
        option src_ipset '1'

config include
        option path '/etc/vpn-policy-routing.netflix.user'
        option enabled '0'

config include
        option path '/etc/vpn-policy-routing.aws.user'
        option enabled '0'

I'd suggest to remove the vpn-pbr completely (and the configs), reinstall from scratch and add only the rule for the site you want to bypass.

Dominik-1980 · January 3, 2021, 2:09am

It works!!

Here is my current config


config policy
        option name 'meineip2'
        option dest_addr 'www.wieistmeineip.de'
        option interface 'lan'

config policy
        option dest_addr 'wieistmeineip.de'
        option name 'meineip'
        option interface 'lan'

config vpn-policy-routing 'config'
        option ipv6_enabled '0'
        option boot_timeout '30'
        option webui_sorting '1'
        list webui_supported_protocol 'tcp'
        list webui_supported_protocol 'udp'
        list webui_supported_protocol 'tcp udp'
        list webui_supported_protocol 'icmp'
        list webui_supported_protocol 'all'
        option webui_enable_column '1'
        option webui_protocol_column '1'
        option webui_chain_column '1'
        list ignored_interface 'vpnserver wgserver'
        option iprule_enabled '0'
        option verbosity '1'
        option strict_enforcement '1'
        option enabled '1'
        option iptables_rule_option 'append'
        option dest_ipset 'ipset'
        option src_ipset '1'
        list supported_interface 'tun0'
        list supported_interface 'lan'

config include
        option path '/etc/vpn-policy-routing.netflix.user'
        option enabled '0'

config include
        option path '/etc/vpn-policy-routing.aws.user'
        option enabled '0'

I think it was the option src_ipset which was 0 on my previous config, now set to 1.

Dominik-1980 · January 3, 2021, 2:21am

@trendy I am new to this forum.
Is there any way to donate you something.
I don’t want to get all this great support for free...

Dominik-1980 · January 3, 2021, 2:29am

I was too fast... It stopped working... How can this be?
It definitely worked and some minutes later it stopped working.

trendy · January 3, 2021, 2:42am

Let's better sleep and check it again tomorrow
The only thing I can imagine is that some IP was used that wasn't added in the list before.

You may donate to the project!

Dominik-1980 · January 3, 2021, 2:46am

Okay. See you tomorrow.
It tried it again, copy and paste my working config. Restart vpn-policy-routing and dnsmasq. It works. 2 minutes later it stopped working... Nothing visible in syslog.
Possibly something overrides the ip set.