Wireguard bypass for specific application

[lleachii]

@elgatito

have added rules in this way:

Did the IP rules by dport fail that I suggested?

I'm wondering why fwmark was used instead?

[elgatito]

@lleachii

Did the IP rules by dport fail that I suggested?

I'm wondering why fwmark was used instead?

@lleachii

If you want to only use it for certain ports:

• you'll have to make an ip table with a default routing rule using WG
• you'll have to make a ip rule for dport to use that rule
• you may also need to make sport ip rule for the reply traffic

Sorry, I'm not quite understanding what is needed to execute here...
Something like ip rule add sport 6881-6899 lookup 202 ?

[lleachii]

@elgatito

Something like ip rule add sport 6881-6899 lookup 202 ?

That's what I was thinking...although, I didn't verify that ranges are valid syntax from the manual.

Did it work?

[elgatito]

@lleachii

That's what I was thinking...although, I didn't verify that ranges are valid syntax from the manual.

Did it work?

At that moment I did not understand what you wrote and what is needed..
I will again with ip rule.

[elgatito]

@lleachii

That's what I was thinking...although, I didn't verify that ranges are valid syntax from the manual.

"ip rule add sport" can not work as expected

https://man.cx/ip-rule ip rule { add | del } SELECTOR ACTION ... SELECTOR := ... [ sport [ NUMBER | NUMBER - NUMBER ] ] [ dport [ NUMBER | NUMBER - NUMBER ] ]

sport/dport are not supported here...

[elgatito]

I did try to make rules for 80 and 443 ports, to see if that works in a browser.

iptables -t mangle -A PREROUTING -p tcp --sport 443 -j MARK --set-xmark 0xc8
iptables -t mangle -A PREROUTING -p tcp --dport 443 -j MARK --set-xmark 0xc8
iptables -t mangle -A PREROUTING -p tcp --sport 80 -j MARK --set-xmark 0xc8
iptables -t mangle -A PREROUTING -p tcp --dport 80 -j MARK --set-xmark 0xc8

ip rule add fwmark 0xc8 lookup 202

And I see old (wan) ip address is used to access websites.

My firewall config:

# cat /etc/config/firewall 

config defaults
	option syn_flood '1'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option flow_offloading '1'
	option flow_offloading_hw '1'

config zone
	option name 'lan'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'
	option network 'lan'

config zone
	option name 'wan'
	option input 'REJECT'
	option output 'ACCEPT'
	option network 'wan wan6'
	option masq '1'
	option mtu_fix '1'
	option forward 'ACCEPT'

config rule
	option name 'Allow-DHCP-Renew'
	option src 'wan'
	option proto 'udp'
	option dest_port '68'
	option target 'ACCEPT'
	option family 'ipv4'

config rule
	option name 'Allow-Ping'
	option src 'wan'
	option proto 'icmp'
	option icmp_type 'echo-request'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-IGMP'
	option src 'wan'
	option proto 'igmp'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-DHCPv6'
	option src 'wan'
	option proto 'udp'
	option src_ip 'fc00::/6'
	option dest_ip 'fc00::/6'
	option dest_port '546'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-MLD'
	option src 'wan'
	option proto 'icmp'
	option src_ip 'fe80::/10'
	list icmp_type '130/0'
	list icmp_type '131/0'
	list icmp_type '132/0'
	list icmp_type '143/0'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Input'
	option src 'wan'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	list icmp_type 'router-solicitation'
	list icmp_type 'neighbour-solicitation'
	list icmp_type 'router-advertisement'
	list icmp_type 'neighbour-advertisement'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Forward'
	option src 'wan'
	option dest '*'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-IPSec-ESP'
	option src 'wan'
	option dest 'lan'
	option proto 'esp'
	option target 'ACCEPT'

config rule
	option name 'Allow-ISAKMP'
	option src 'wan'
	option dest 'lan'
	option dest_port '500'
	option proto 'udp'
	option target 'ACCEPT'

config include
	option path '/etc/firewall.user'

config zone
	option output 'ACCEPT'
	option name 'wgzone'
	option input 'REJECT'
	option masq '1'
	option mtu_fix '1'
	option network 'wg0'
	option forward 'ACCEPT'

config forwarding
	option dest 'wan'
	option src 'lan'

config forwarding
	option dest 'wgzone'
	option src 'lan'

config forwarding
	option dest 'lan'
	option src 'wgzone'

config include 'miniupnpd'
	option type 'script'
	option path '/usr/share/miniupnpd/firewall.include'
	option family 'any'
	option reload '1'

I have a wgzone and forwardings coming to it, but still something is missing for me ...

[elgatito]

I did also try with Vpn-Policy-Routing app.
Added rule to route local/remote 443 through wan interface, firewall shows this:

Chain VPR_PREROUTING (References: 1)
Pkts.	Traffic	Target	Prot.	In	Out	Source	Destination	Options
138	8.00 KB	MARK	tcp	*	*	0.0.0.0/0	0.0.0.0/0	multiport dports 443 /* ssl-to */ MARK xset 0x10000/0xff0000
265	14.39 KB	MARK	tcp	*	*	0.0.0.0/0	0.0.0.0/0	multiport sports 443 /* ssl */ MARK xset 0x10000/0xff0000
0	0.00 B	MARK	tcp	*	*	0.0.0.0/0	0.0.0.0/0	multiport dports 123 /* ntp */ MARK xset 0x10000/0xff0000
0	0.00 B	MARK	all	*	*	0.0.0.0/0	0.0.0.0/0	match-set wg0 dst MARK xset 0x20000/0xff0000
28	2.30 KB	MARK	all	*	*	0.0.0.0/0	0.0.0.0/0	match-set wan dst MARK xset 0x10000/0xff0000

But when I try to open any https website - I get a connection timeout.

[ulmwind]

Something like offtop, in another case we haven't managed to configure vpn-policy-routing. Could you give output of:
iptables -S -t mangle | grep VPR_PRE

[elgatito]

@ulmwind

iptables -S -t mangle | grep VPR_PRE

Here you go

root@OpenWrt:~# iptables -S -t mangle | grep VPR_PRE
-N VPR_PREROUTING
-A PREROUTING -m mark --mark 0x0/0xff0000 -j VPR_PREROUTING
-A VPR_PREROUTING -p tcp -m multiport --dports 443 -m comment --comment ssl-to -j MARK --set-xmark 0x10000/0xff0000
-A VPR_PREROUTING -p tcp -m multiport --sports 443 -m comment --comment ssl -j MARK --set-xmark 0x10000/0xff0000
-A VPR_PREROUTING -p tcp -m multiport --dports 123 -m comment --comment ntp -j MARK --set-xmark 0x10000/0xff0000
-A VPR_PREROUTING -m set --match-set wg0 dst -j MARK --set-xmark 0x20000/0xff0000
-A VPR_PREROUTING -m set --match-set wan dst -j MARK --set-xmark 0x10000/0xff0000

and the full mangle table

root@OpenWrt:~# iptables -S -t mangle
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
-N VPR_FORWARD
-N VPR_INPUT
-N VPR_OUTPUT
-N VPR_PREROUTING
-A PREROUTING -m mark --mark 0x0/0xff0000 -j VPR_PREROUTING
-A INPUT -m mark --mark 0x0/0xff0000 -j VPR_INPUT
-A FORWARD -m mark --mark 0x0/0xff0000 -j VPR_FORWARD
-A FORWARD -o br-lan -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment --comment "!fw3: Zone lan MTU fixing" -j TCPMSS --clamp-mss-to-pmtu
-A FORWARD -o eth0.2 -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment --comment "!fw3: Zone wan MTU fixing" -j TCPMSS --clamp-mss-to-pmtu
-A FORWARD -o wg0 -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment --comment "!fw3: Zone wgzone MTU fixing" -j TCPMSS --clamp-mss-to-pmtu
-A OUTPUT -m mark --mark 0x0/0xff0000 -j VPR_OUTPUT
-A VPR_PREROUTING -p tcp -m multiport --dports 44300 -m comment --comment ssl-to -j MARK --set-xmark 0x10000/0xff0000
-A VPR_PREROUTING -p tcp -m multiport --sports 44300 -m comment --comment ssl -j MARK --set-xmark 0x10000/0xff0000
-A VPR_PREROUTING -p tcp -m multiport --dports 123 -m comment --comment ntp -j MARK --set-xmark 0x10000/0xff0000
-A VPR_PREROUTING -m set --match-set wg0 dst -j MARK --set-xmark 0x20000/0xff0000
-A VPR_PREROUTING -m set --match-set wan dst -j MARK --set-xmark 0x10000/0xff0000

[ulmwind]

OK, it should work for forwarded traffic. Could you add port 22 TCP, and try to login to some server by SSH, and see you IP?

[fatbob313]

Don't know if this helps, but I had a similiar problem where I wanted a specific application running on the router itself have its traffic routed through the wireguard interface. Banged my head against policy based routing for a while but ended up using network namespace instead which actually is simpler to set up (in my opinion). The background is explained here: https://www.wireguard.com/netns/

To set up a namspace called 'container' and create and move a wireguard interface to it:

ip netns add container
ip netns exec container ip link set dev lo up
ip netns exec container echo "nameserver <Wireguard provider DNS 1>" > /etc/resolv.conf
ip netns exec container echo "nameserver <Wireguard provider DNS 2>" >> /etc/resolv.conf
ip link add dev wg0 type wireguard
ip link set wg0 netns container

ip -n container addr add <ip address of wg0 interface>  dev wg0
ip netns exec container wg setconf wg0 /etc/wireguard/wg0.conf

ip -n container link set wg0 up
ip -n container route add default dev wg0

Note that the wireguard conf-file must have the endpoint's IP-address.

To test it start a shell in the container namespace:

ip netns exec container sh
ifconfig
ping www.google.com

[elgatito]

@ulmwind

OK, it should work for forwarded traffic. Could you add port 22 TCP, and try to login to some server by SSH, and see you IP?

When I enable "route_allowed_ips" in wireguard, and all the traffic goes through wg interface, then I set rule to route 443 port (incoming/outcoming) through wan interface - I get connection timeouts in the browser.
Same thing when everything comes through wan and I try to route 443 through wireguard.

For me it looks like firewall is blocking it, or it cannot route where needed.

[ulmwind]

Could you check, or not?

[Naftali]

How to this in LuCI GUI with "VPN Policy Routing"?
Can somebody post screenshots of a working setup?

[lleachii]

@Naftali

How to this in LuCI GUI with "VPN Policy Routing"?

As noted above, directing ports cannot be done by route policy (on default software).

(You may wish to make a new thread, and explain what you're trying to setup.)

[elgatito]

I'm writing again about this issue.

I have two possible ways to split usage of Wireguard:

1. Wireguard is enabled by default by enabling "route_allowed_ips", ports 80 and 443 are added to VPN Policy Routing to go through WAN.
2. Wireguard is not the default route, ports, used by desired application are added to VPN Policy Routing to go through WG0.

I have Openwrt 19.07.3 (installed today), with all the modules also being up-to-date.
I have tried both ways. The one (1) with 80/443 exception looks simplier, but any way did not work for me.

I see in Firewall status page of Luci that rules are matched (PREROUTING section).
But it looks like everything is firewalled, browser pages are not opening.

When trying the (2) way - that application is not able to open connections.

So in both scenarios I'm not able to use non-default route.

Maybe there is something stupid-simple to check for?

[elgatito]

Found somewhere advises to replace allowed_ips in wireguard:
0.0.0.0/0 to -> 0.0.0.0/1 + 128.0.0.0/1

Did that and still not helping.
If "route_allowed_ips" is enabled - I cannot access policies that go to WAN.

[vgaetera]

@elgatito

0.0.0.0/1 + 128.0.0.0/1

This is the recommended method as it helps restore the WAN gateway if you decide to stop the WG.

@elgatito

If "route_allowed_ips" is enabled - I cannot access policies that go to WAN.

https://github.com/stangri/openwrt_packages/blob/master/vpn-policy-routing/files/README.md
Service does not alter the default routing. Depending on your VPN tunnel settings (and settings of the VPN server you are connecting to), the default routing might be set to go via WAN or via VPN tunnel. This service affects only routing of the traffic matching the policies. If you want to override default routing, follow the instructions below.

option route_allowed_ips '0'

[elgatito]

I did try with "route_allowed_ips=0".
Then setting to go ports 80 and 443 through wg0, and then getting nothing in the browser, not connecting to any website.

[vgaetera]

uci show network; uci show firewall; uci show vpn-policy-routing