Static IP device - static routes bypass routing table

otnert · August 19, 2018, 2:26am

I'm using 17.01.5 on my router and it's configured using a VPN for all devices, I've also configured it so a statically assigned IP device bypasses the VPN and routes through using my ISP IP address and DNS (working fine).

However, when I create a static route (working to some degree), the static IP device bypasses the static route, yet all VPN clients use the static route.

The setup:

I installed the ip package
created a routing table 10 exvpn in ' /etc/iproute2/rt_tables '

128 prelocal
255 local
254 main
253 default
10 exvpn
0 unspec

added host rules, route to the new table in ' /etc/rc.local ' where:
eth0 is the OpenWrt WAN interface
192.168.2.1 OpenWrt router IP
192.168.10.1 is my ISP modem IP, where I have the OpenWrt router connected to

ip rule add from 192.168.2.253 table exvpn
ip route add 192.168.2.253 dev eth0 table exvpn
ip route add default via 192.168.10.1 dev eth0 table exvpn
ip route flush cache
exit 0

added a static IP 192.168.2.253 under >Network >DHCP and DNS >Static Leases for my device

As I mentioned above the static IP 192.168.2.253 device bypasses the static route, all VPN devices don't.

I want to have the static IP device also utilize the static route but I'm not sure how to do this?

lleachii · August 19, 2018, 5:11am

Note, that you can add routes and rules to /etc/config/network and via the UCI.

See this (it's not specific to Wireguard):

otnert · August 20, 2018, 1:46am

@lleachii My needs are very basic, possibly 1-5 devices I assign a static IP, to bypass my VPN.

It seems no matter what I do, if I have the routing table, Static Route(s) (using >Network >Static Routes >Static IPv4 Routes) and Static Lease set for 192.168.2.253 as described above, the static routes are bypassed for the statically assigned IP(s). Is this by design i.e. routes over and above the router assigned routing table defaults are bypassed when using this setup?

OTOH...
If I setup a Firewall rule e.g. iptables -I FORWARD -d 1.2.3.4/255.255.255.255 -j REJECT this seems to work if I'm using the Static (exVPN) devices or DHCP allocated (VPN) devices.

lleachii · August 20, 2018, 7:05am

I'm not quite sure what issue you're having here. If you want 192.168.2.253 to use WAN, configure it to do so. I find it easier to keep my WAN as the default gateway, but you can configure it as you prefer. It seems you configured 192.168.2.253 to bypass your VPN, now you wish to disable it?

If you don't wish to follow the guide that configures this using subnets, just use individual /32 IP addresses and repeat the process for each one.

I have no clue of the purpose of this firewall rule.

otnert · August 20, 2018, 8:05am

Any static IP devices configured to bypass the VPN + static routes = static routes not working.
Any VPN devices = static routes work.

To take the place of the static routes that don't seem to work = no issue for both WAN & VPN devices.

lleachii · August 20, 2018, 8:32am

Show the rule for that bypassed IP
Show the route/table config for it

jow · August 20, 2018, 8:36am

Note that the act of adding a static route will restart netifd which in turn will flush the IP rules set up by rc.local. To prevent that, you should either encode both your ip rules and your routes into /etc/config/network or setup both the rules and routes in /etc/rc.local (less preferable).

otnert · August 20, 2018, 12:42pm

Hope this makes things clearer

result:

config route
        option interface 'lan'
        option target '8.8.8.8'
        option netmask '255.255.255.255'
        option gateway '192.168.2.1'
        option metric '2'

config route
        option interface 'lan'
        option target '8.8.4.4'
        option netmask '255.255.255.255'
        option gateway '192.168.2.1'
        option metric '2'

config route
        option interface 'lan'
        option target '108.175.32.0'
        option netmask '255.255.240.0'
.
.

VPN - configured as router default - (DHCP)
RESULT

exVPN above - part of the exVPN routing table - (Static IP)

The static routes are bypassed for all the IP's listed in the exVPN routing table per the replies from 8.8.8.8
I've rebooted the router with the same result.

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

I've ONLY used IPTABLES below as last measure to highlight the issue with the static routes.

result:
RESULT

jow · August 20, 2018, 12:44pm

Please provide the output of ip -4 route show table all and ip -4 rule in the broken state.

otnert · August 20, 2018, 1:00pm

Hi @jow thanks for looking at this for me, appreciate it!

There are only 2x static routes shown, even though I normally have more but used for testing ATM.

  \    LE  \    /  -----------------------------------------------------------
   \  DE    \  /    Reboot (17.01.5, r3919-38e704be71)
    \________\/    -----------------------------------------------------------

root@LEDE:~# ip -4 route show table all
default via 192.168.10.1 dev eth0  table exvpn
192.168.2.250 dev eth0  table exvpn  scope link
192.168.2.251 dev eth0  table exvpn  scope link
192.168.2.252 dev eth0  table exvpn  scope link
192.168.2.253 dev eth0  table exvpn  scope link
192.168.2.254 dev eth0  table exvpn  scope link
0.0.0.0/1 via 10.24.10.5 dev tun0
default via 192.168.10.1 dev eth0  proto static  src 192.168.10.4
8.8.4.4 via 192.168.2.1 dev br-lan  proto static  metric 2
8.8.8.8 via 192.168.2.1 dev br-lan  proto static  metric 2
10.24.10.1 via 10.24.10.5 dev tun0
10.24.10.5 dev tun0  proto kernel  scope link  src 10.24.10.6
128.0.0.0/1 via 10.24.10.5 dev tun0
168.1.99.215 via 192.168.10.1 dev eth0
192.168.2.0/24 dev br-lan  proto kernel  scope link  src 192.168.2.1
192.168.3.0/24 dev br-guest  proto kernel  scope link  src 192.168.3.1
192.168.10.0/24 dev eth0  proto kernel  scope link  src 192.168.10.4
192.168.10.1 dev eth0  proto static  scope link  src 192.168.10.4
local 10.24.10.6 dev tun0  table local  proto kernel  scope host  src 10.24.10.6
broadcast 127.0.0.0 dev lo  table local  proto kernel  scope link  src 127.0.0.1
local 127.0.0.0/8 dev lo  table local  proto kernel  scope host  src 127.0.0.1
local 127.0.0.1 dev lo  table local  proto kernel  scope host  src 127.0.0.1
broadcast 127.255.255.255 dev lo  table local  proto kernel  scope link  src 127.0.0.1
broadcast 192.168.2.0 dev br-lan  table local  proto kernel  scope link  src 192.168.2.1
local 192.168.2.1 dev br-lan  table local  proto kernel  scope host  src 192.168.2.1
broadcast 192.168.2.255 dev br-lan  table local  proto kernel  scope link  src 192.168.2.1
broadcast 192.168.3.0 dev br-guest  table local  proto kernel  scope link  src 192.168.3.1
local 192.168.3.1 dev br-guest  table local  proto kernel  scope host  src 192.168.3.1
broadcast 192.168.3.255 dev br-guest  table local  proto kernel  scope link  src 192.168.3.1
broadcast 192.168.10.0 dev eth0  table local  proto kernel  scope link  src 192.168.10.4
local 192.168.10.4 dev eth0  table local  proto kernel  scope host  src 192.168.10.4
broadcast 192.168.10.255 dev eth0  table local  proto kernel  scope link  src 192.168.10.4
root@LEDE:~#
root@LEDE:~# ip -4 rule
0:      from all lookup local
32761:  from 192.168.2.254 lookup exvpn
32762:  from 192.168.2.253 lookup exvpn
32763:  from 192.168.2.252 lookup exvpn
32764:  from 192.168.2.251 lookup exvpn
32765:  from 192.168.2.250 lookup exvpn
32766:  from all lookup main
32767:  from all lookup default

lleachii · August 20, 2018, 3:06pm

Make an ip rule for the static IP to use this route

config rule
	option src '192.168.2.253/32'
	option dest '0.0.0.0/0'
	option priority '2'
	option lookup 'main'

otnert · August 21, 2018, 3:51am

Hi @lleachii I've implemented the ip rule & rebooted, it looks like the rule has taken hold, however it is still exhibiting the same behaviour

root@LEDE:~# ip -4 rule
0:      from all lookup local
0:      from 192.168.2.251 lookup exvpn
0:      from 192.168.2.252 lookup exvpn
0:      from 192.168.2.253 lookup exvpn
0:      from 192.168.2.254 lookup exvpn
1:      from 192.168.2.250 lookup exvpn
2:      from 192.168.2.252 lookup main
2:      from 192.168.2.253 lookup main
32766:  from all lookup main
32767:  from all lookup default

IP's included in the 'exvpn' routing table (192.168.2.250-254) come back with replies:
Reply from 8.8.8.8: bytes=32 time=33ms TTL=121
Other IP's (DHCP) come back with:
Reply from 192.168.2.1: Destination host unreachable.

root@LEDE:~# ip -4 route show table all
default via 192.168.10.1 dev eth0  table exvpn
192.168.2.250 dev eth0  table exvpn  scope link
192.168.2.251 dev eth0  table exvpn  scope link
192.168.2.252 dev eth0  table exvpn  scope link
192.168.2.253 dev eth0  table exvpn  scope link
192.168.2.254 dev eth0  table exvpn  scope link
0.0.0.0/1 via 10.12.10.5 dev tun0
default via 192.168.10.1 dev eth0  proto static  src 192.168.10.4
8.8.4.4 via 192.168.2.1 dev br-lan  proto static  metric 2
8.8.8.8 via 192.168.2.1 dev br-lan  proto static  metric 2
10.12.10.1 via 10.12.10.5 dev tun0
10.12.10.5 dev tun0  proto kernel  scope link  src 10.12.10.6
128.0.0.0/1 via 10.12.10.5 dev tun0
168.1.75.24 via 192.168.10.1 dev eth0
192.168.2.0/24 dev br-lan  proto kernel  scope link  src 192.168.2.1
192.168.3.0/24 dev br-guest  proto kernel  scope link  src 192.168.3.1
192.168.10.0/24 dev eth0  proto kernel  scope link  src 192.168.10.4
192.168.10.1 dev eth0  proto static  scope link  src 192.168.10.4
local 10.12.10.6 dev tun0  table local  proto kernel  scope host  src 10.12.10.6
broadcast 127.0.0.0 dev lo  table local  proto kernel  scope link  src 127.0.0.1
local 127.0.0.0/8 dev lo  table local  proto kernel  scope host  src 127.0.0.1
local 127.0.0.1 dev lo  table local  proto kernel  scope host  src 127.0.0.1
broadcast 127.255.255.255 dev lo  table local  proto kernel  scope link  src 127.0.0.1
broadcast 192.168.2.0 dev br-lan  table local  proto kernel  scope link  src 192.168.2.1
local 192.168.2.1 dev br-lan  table local  proto kernel  scope host  src 192.168.2.1
broadcast 192.168.2.255 dev br-lan  table local  proto kernel  scope link  src 192.168.2.1
broadcast 192.168.3.0 dev br-guest  table local  proto kernel  scope link  src 192.168.3.1
local 192.168.3.1 dev br-guest  table local  proto kernel  scope host  src 192.168.3.1
broadcast 192.168.3.255 dev br-guest  table local  proto kernel  scope link  src 192.168.3.1
broadcast 192.168.10.0 dev eth0  table local  proto kernel  scope link  src 192.168.10.4
local 192.168.10.4 dev eth0  table local  proto kernel  scope host  src 192.168.10.4
broadcast 192.168.10.255 dev eth0  table local  proto kernel  scope link  src 192.168.10.4

lleachii · August 21, 2018, 9:18am

Your IP rules have the same priority number. I'm not sure if you did this on purpose.

otnert · August 21, 2018, 9:31am

I take it you mean the four 0: numbers followed by the single 1: number exvpn rule (which is also one of mine). The system has allocated the rule numbering.

lleachii · August 21, 2018, 9:38am

Incorrect.

This is the cause of the rule numbering.

otnert · August 21, 2018, 10:56am

It would be advantageous if I knew exactly what I'm supposed to be looking at here, apart from ? some type of priority rule numbering/order I need to be following, as I have no idea ATM

jow · August 21, 2018, 11:12am

What I am not understanding about your setup; why are you adding the 192.168.2.251/32 .. 192.168.2.253/32 host routes into the exvpn routing table?

When I did policy routing setups in the past I merely put an alternative sole default route into the extra routing table, no extra host routes for each IP.

Can you for completeness also provide your current complete /etc/config/network?

This would be my setup (in /e/c/network):

# Stage the alternative default route in table exvpn
config route
  option interface wan  # I assume the wan is connected to your modem
  option target 0.0.0.0
  option netmask 0.0.0.0
  option gateway 192.168.10.1
  option table exvpn

# Direct clients 192.168.2.250 - 192.168.2.254 to table exvpn
config rule
  option in lan
  option src 192.168.2.250
  option lookup exvpn

config rule
  option in lan
  option src 192.168.2.251
  option lookup exvpn

config rule
  option in lan
  option src 192.168.2.252
  option lookup exvpn

config rule
  option in lan
  option src 192.168.2.253
  option lookup exvpn

config rule
  option in lan
  option src 192.168.2.254
  option lookup exvpn

otnert · August 21, 2018, 11:25am

OK so if I change my set as how you have described, will the 192.168.2.250-254 IP's bypass the VPN (my preferred router default)?

This is the reason as to why I'm doing this, I want the 5x IP's to have my ISP modem address not my VPN address.
... and of course require the static routes to apply to these 5x IP's as they haven't been

jow · August 21, 2018, 11:29am

Yes, they will (or should, rather )

I suggest to try the additions above, then remove any manual ip rule / ip route commands, maybe perform a reboot to be sure. Then again test the route, e.g. by running mtr or traceroute once on a vpn using host and once on one of the five excempted ones.

If it does not work, please re-post ip -4 rule show; ip -4 route show table all.

A specific /32 route per host should definitely not be required. You only need the one alternative default route and one or more ip rules to direct traffic into that separate table.

otnert · August 21, 2018, 11:30am

OK ! onto it now!