Routing problem, unable to force a specific indirect route

Hi,

I'm working to prepare a keepalived config, but first I needed to create an intermediate route that one day may be a VIP (I have 3 wi-fi route in the house, they crash often). But I cannot even route correctly.

I have a LAN zone 192.168.0.0/24 (br-lan.1) which is the home servers, and I want to make a guest network on 192.168.20.0/32 (br-lan.10) routed to an interface "guestha" 192.168.10.1 (br-lan.10) then to the network LAN via firewall masqueradins, to the ISP router 192.168.0.254.

It works well, except that If I set the route to a wrong address (say 192.168.10.7 instead of .1) it continues to work perfectly. traceroute says that I jump from 192.168.20.1 to 192.168.0.254 (the ISP router).

I've put route
0.0.0.0/32 -> 192.168.10.1 on "guest" (192.168.20.0/24)
and reverse route
192.168.20.0/24 -> 192.168.10.1 on guestha(192.168.10.0/24)

and other routes from lan (in case)
but it seems that it ignores my routes.

tracert see a jump to the final gateway, and if I put
1 2 ms 2 ms 8 ms 192.168.20.1
2 2 ms 1 ms 1 ms 192.168.0.254

more surprising, I don't see my routes in the "ip route show table all"

0.0.0.0 via 192.168.0.254 dev br-lan.1  src 192.168.0.251  metric 2
default via 192.168.0.254 dev br-lan.1  src 192.168.0.251
192.168.0.0/24 dev br-lan.1 scope link  src 192.168.0.251
192.168.10.0/24 dev br-lan.10 scope link  src 192.168.10.1
192.168.20.0/24 dev br-lan.11 scope link  src 192.168.20.1
local 127.0.0.0/8 dev lo table local scope host  src 127.0.0.1
local 127.0.0.1 dev lo table local scope host  src 127.0.0.1
broadcast 127.255.255.255 dev lo table local scope link  src 127.0.0.1
local 192.168.0.251 dev br-lan.1 table local scope host  src 192.168.0.251
broadcast 192.168.0.255 dev br-lan.1 table local scope link  src 192.168.0.251
local 192.168.10.1 dev br-lan.10 table local scope host  src 192.168.10.1
broadcast 192.168.10.255 dev br-lan.10 table local scope link  src 192.168.10.1
local 192.168.20.1 dev br-lan.11 table local scope host  src 192.168.20.1
broadcast 192.168.20.255 dev br-lan.11 table local scope link  src 192.168.20.1

I really don't understand.
I've tried to add metric value, I've added source address (is it useful?)... but it seems to ignore my rules, and moreover to route straight to the default gateway.
I've searched for similar problem, some consider puting metrics at value 2...
I've set the default gateway in both interfaces, to put 0.0.0.0...
but nothing change. Sure I missed a point.

here is my /etc/config/network (without the route6)

config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fd17:b4b5:1c96::/48'
	option packet_steering '1'

config device
	option name 'br-lan'
	option type 'bridge'
	list ports 'lan1'
	list ports 'lan2'
	list ports 'lan3'
	list ports 'lan4'
	option stp '1'
	option igmp_snooping '1'

config interface 'lan'
	option device 'br-lan.1'
	option proto 'dhcp'
	option delegate '0'

config interface 'lan6'
	option device 'br-lan.1'
	option proto 'dhcpv6'
	option reqprefix 'auto'
	option reqaddress 'try'
	option delegate '0'

config bridge-vlan
	option device 'br-lan'
	option vlan '1'
	list ports 'lan1:u*'
	list ports 'lan2:u*'
	list ports 'lan3:u*'

config bridge-vlan
	option device 'br-lan'
	option vlan '10'
	list ports 'lan1:t'

config bridge-vlan
	option device 'br-lan'
	option vlan '11'
	list ports 'lan4:u*'

config interface 'guest'
	option proto 'static'
	option device 'br-lan.11'
	option ipaddr '192.168.20.1'
	option netmask '255.255.255.0'
	option ip6assign '64'
	list ip6class 'local'
	option ip6ifaceid '::1'
	option ip6hint '20'
	option gateway '192.168.10.1'

config interface 'guestha'
	option proto 'static'
	option device 'br-lan.10'
	option delegate '0'
	option ip6assign '64'
	option ip6hint '10'
	list ip6class 'local'
	option ip6ifaceid '::1'
	option ipaddr '192.168.10.1'
	option netmask '255.255.255.0'
	option gateway '192.168.0.254'

config route
	option interface 'guest'
	option target '0.0.0.0/32'
	option gateway '192.168.10.1'
	option metric '2'
	option source '192.168.20.1'

config route
	option interface 'guestha'
	option target '192.168.20.0/24'
	option gateway '192.168.20.1'
	option metric '2'
	option source '192.168.10.1'

config route
	option interface 'lan'
	option target '0.0.0.0/32'
	option gateway '192.168.0.254'
	option metric '2'
	option source '192.168.0.251'

config route
	option interface 'lan'
	option target '192.168.20.0/24'
	option gateway '192.168.10.1'
	option metric '1'
	option source '192.168.10.1'

config route
	option interface 'lan'
	option target '192.168.10.0/24'
	option gateway '192.168.10.1'
	option source '192.168.10.1'
	option metric '2'

Thanks in advance.

I have a hypothesis, which imply I'm dumb :woozy_face:. I investigate it.

the idea is that I'm trying to add a route from br-lan.11 toward a gateway which is only on br-lan.10, and since there is no bridging, route is refused,, it passes through the router and perfectly works.

the problem is that I don't understand well the way it works, between network segment connected to the same OpenWrt router via different interfaces.

Why? What are you trying to achieve as an end goal?

My goal was to prepare for keepalived, to test a way to redirect the traffic from local wifi clients toward a single openwrt router firewall that does masquerading.My routers crash often, and clients are roaming with 802.1r... (and it's a challenge for me, say it's hobby...)

I have seen that keepalived create a virtual IP, so I decided to create one to test how to make it work... however I missed key concept...

my vision
router1: wifi client->guest interface -> guestha VIP
router2: guestha VIP -> masquerade -> LAN

my current toy test:
router1: wifi client->guest interface -> guestHA static IP -> masquerade -> LAN

So when a router acting as an AP crashes how are you expecting keepalived is going to help?

if one AP crash normally another AP should allow the client to roam using 802.1r.
I hope so. and the VIP should move to a working node...
Sure I should test.

Why do you think keepalived is necessary?

Running two IP subnets on the same L2 segment (VLAN) is not a standard practice, because it doesn't do anything practical. There is no way to force traffic to go through a router when a direct path exists. The OS will always make an ARP for the direct path ("who has 192.168.10.7, tell 192.168.20.5") and if that is answered, 192.168.20.5 will use the MAC address of 192.168.10.7 to make a direct link.

OK, now I understand. My idea is wrong.
Whatever I do about routing, it will use the router local interface as a shortcut...

The only way seems to be to publish the floating VIP as DHCP gateway (DHCP Option 3?)...
For IPV6 I don't see a way... I will dig more... maybe it's a known problem.

I'm pretty sure you have no idea what you're doing or why you're doing it. Keepalived is not a solution to crashing APs, nor is it necessary for WiFi clients moving between APs.

You are a bit right, but I've learned much.
Here are my findings.

I had greatly misunderstood the way routes works, the interface meaning.
As I understand now, interface in a route mean the interface responsible to relay the packet to the gateway (not the one receiving the packet as I imagined).
I also understand now that all interfaces are just plugged on the router "routing hub", not wired like a circuit like I wanted... the routing hub try to find a route that matches the packet received on an interface, and send it to the interface found by routing tables that will relay to the gateway on it's link...

I could not add some routes because it was just absurd.

today it works with DHCP.
I removed the br-lan.11, the guestha interface...
the guest interface are on a VLAN shared by the routers, each with an IP.
192.168.10.1|2|3
keepalived created a VIP 192.168.10.20 that is just plugged to the routing hub (the firewall too...) like any interface is... I can use it as gateway, as dns...
I just added DHCP-Option 3 and 6.

I will see what to do with IPV6 RA... There also I had to correct many misconceptions, but sure I'm still learning.

It still sounds like you're massively over-complicating things. There should be no need to be using keepalived or messing with routing to have WiFi clients switch AP using fast roaming.

What is the setup of your full network? What is connected to what?

Right,
It's also experimental. Maybe I will have to step back.
My fear was that when changing of AP (roaming), the masquerade will not be the same on the second AP... but it's probably not a problem for normal user (mobile phone or laptop )...
It's not rational, I agree. I just want to understand.

You generally wouldn't have APs within a network doing any sort of masquerading. If you do then you're probably doing it wrong. You would tend to do masquerading at the boundary of the network if there's a need to hide or transform the source IP (for most home users it's when you're going from multiple private internal IPs to a singular public IP).

Masquerading is used because Guest client are on a non public network, IPv4 and IPv6.
It cannot be routed to the Internet without that. This part works well.
But yes, keepalived is overkill there.

Having looked at some of your posting history, it appears your setup works more through luck than anything else. We can assist with checking and correcting, but we'd need to see the full configs for all the OpenWRT devices as well as details of the overall network setup.

That's because I don't catch the keys yet... I'm learning.
My first question here was just based on deep misconceptions, thanks for correcting them.

Now it works well in IPv4, It's simply a local network with DHCPv4 and masquerading to the WAN... I just changed the DHCP gateway to the keepalived IP... It works alone for the moment.

IPv6 works without keepalived. RA service publish the router's address, and there is ipv6 masquerading that works well...

I need to clean up all, because, yes, there is some luck in it, but I progress.

Here are some of my config

network


config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fd17:b4b5:1c96::/48'
	option packet_steering '1'

config device
	option name 'br-lan'
	option type 'bridge'
	list ports 'lan1'
	list ports 'lan2'
	list ports 'lan3'
	list ports 'lan4'
	option stp '1'
	option igmp_snooping '1'

config interface 'lan'
	option device 'br-lan.1'
	option proto 'dhcp'
	option delegate '0'

config interface 'lan6'
	option device 'br-lan.1'
	option proto 'dhcpv6'
	option reqprefix 'auto'
	option reqaddress 'try'
	option delegate '0'

config bridge-vlan
	option device 'br-lan'
	option vlan '1'
	list ports 'lan1:u*'
	list ports 'lan2:u*'
	list ports 'lan3:u*'

config bridge-vlan
	option device 'br-lan'
	option vlan '10'
	list ports 'lan1:t'
	list ports 'lan4:u*'

config interface 'guest'
	option proto 'static'
	option device 'br-lan.10'
	option ipaddr '192.168.10.2'
	option netmask '255.255.255.0'
	list ip6class 'local'
	option ip6ifaceid '::2'
	option gateway '192.168.0.254'
	option delegate '0'
	option ip6assign '64'
	option ip6hint '10'

config route6
	option interface 'lan6'
	option target '::/0'
	option gateway '2a01:xxx:yyy:zzzz::1'

config route
	option interface 'lan'
	option target '0.0.0.0/32'
	option gateway '192.168.0.254'
	option source '192.168.10.1'
	option metric '0'

dhcp


config dnsmasq
	option domainneeded '1'
	option localise_queries '1'
	option rebind_protection '1'
	option rebind_localhost '1'
	option local '/lan/'
	option domain 'lan'
	option expandhosts '1'
	option readethers '1'
	option leasefile '/tmp/dhcp.leases'
	option resolvfile '/tmp/resolv.conf.d/resolv.conf.auto'
	option localservice '1'
	option ednspacket_max '1232'

config odhcpd 'odhcpd'
	option maindhcp '0'
	option leasefile '/tmp/hosts/odhcpd'
	option leasetrigger '/usr/sbin/odhcpd-update'
	option loglevel '4'

config dhcp 'guest'
	option interface 'guest'
	option start '96'
	option limit '159'
	option leasetime '30m'
	option ra_useleasetime '1'
	option preferred_lifetime '5m'
	list dhcp_option '3,192.168.10.20'
	list dhcp_option '6,192.168.10.20'
	option ra 'server'
	option ra_default '1'

firewall


config defaults
	option input 'ACCEPT'
	option output 'ACCEPT'
	option synflood_protect '1'
	option forward 'REJECT'
	option drop_invalid '1'

config zone
	option name 'lan'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'DROP'
	option masq '1'
	option masq6 '1'
	list network 'lan'
	list network 'lan6'

config zone
	option name 'guest'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'
	option masq '1'
	option masq6 '1'
	list network 'guest'

config forwarding
	option src 'guest'
	option dest 'lan'

config forwarding
	option src 'lan'
	option dest 'guest'

config rule
	option name 'Allow-Guest-Ping-IPv4-Freebox'
	option src 'guest'
	option dest 'lan'
	option target 'ACCEPT'
	option family 'ipv4'
	list proto 'icmp'
	list dest_ip '192.168.0.254'
	list icmp_type 'echo-request'
	list icmp_type 'extended-echo-request'

config rule
	option name 'Allow-Guest-Ping-IPv6-Freebox'
	option src 'guest'
	option dest 'lan'
	option target 'ACCEPT'
	list dest_ip '2a01:xxx:yyy:::1'
	list proto 'icmpv6'
	option family 'ipv6'
	list icmp_type 'echo-request'
	list icmp_type 'extended-echo-request'

config rule
	option name 'Deny-Guest-All-LAN'
	option src 'guest'
	option dest 'lan'
	option target 'DROP'
	list dest_ip '192.168.0.0/24'
	list dest_ip '2a01:xxx:yyy:zzzz::/64'
	list proto 'all'

keepalived


config globals 'globals'
	option router_id 'R2'

config peer
	option name 'R1'
	option address '192.168.0.250'

config peer
	option name 'R3'
	option address '192.168.0.249'

config track_interface
	option name 'guestlan'
	option value 'br-lan.10'

config vrrp_instance
	option name 'guestha4'
	option state 'MASTER'
	option interface 'br-lan.1'
	option virtual_router_id '4'
	option priority '100'
	option advert_int '1'
	option nopreempt '0'
	option unicast_src_ip '192.168.0.251'
	list unicast_peer 'R1'
	list unicast_peer 'R3'
	option auth_type 'PASS'
	option auth_pass 'vrrp'
	option garp_master_delay '1'
	option garp_master_repeat '1'
	option garp_master_refresh '1'
	option garp_master_refresh_repeat '1'
	list track_interface 'guestlan'
	list virtual_ipaddress 'guestvip4'
	option version '3'

config ipaddress
	option name 'guestvip4'
	option address '192.168.10.20'
	option device 'br-lan.10'
	option label_suffix 'gv4'

config vrrp_instance
	option name 'guestha6'
	option state 'MASTER'
	option interface 'br-lan.10'
	option virtual_router_id '6'
	option priority '100'
	option advert_int '1'
	option nopreempt '0'
	list virtual_ipaddress 'guestvip6l'
	list unicast_peer 'R1'
	list unicast_peer 'R3'
	option auth_type 'PASS'
	option auth_pass 'vrrp'
	list track_interface 'guestlan'
	option garp_master_delay '1'
	option garp_master_repeat '1'
	option garp_master_refresh '1'
	option garp_master_refresh_repeat '1'
	option native_ipv6 '1'
	option version '3'

config ipaddress
	option name 'guestvip6l'
	option address 'fe80::0000:0000:0000:0020'
	option device 'br-lan.10'
	option label_suffix 'vg6l'
	option scope 'link'

NB: It seems ipv4 and ipv6 have to be on separate instances. I've seen that in another thread.

now the only problem if I continue in my (overkill) idea, would be to advertise the ipv6 VIP as the router in RA messages, like I did with DHCPv4

Currently RA advertise my ::2 address from guest interface... it works well, but straight to the local router.

The only ideas I've found, are

  • use uradvd to advertise manually my router VIP
  • use bird which allows to configure RA advertising

uravd seems not well integrated, and bird seems to be a huge subject... rationally I should stop there.

As a starting point just get rid of keepalived. It has it's uses, but not for what you are wanting to do.

Secondly, what is at 192.168.0.254? Is it an ISP router? Is anything else connected to it other than the OpenWRT device? Can it be put in bridge mode so it's functioning as just a modem?

Thirdly, what is OpenWRT running on?

192.168.0.254 is my ISP router. I cannot put it in bridge mode, and thare are NAS in the zone...

If I remove keepalived, in fact, all is working. The roaming is just brutal... I wanted to improve that, but as it is much more trouble than needed...

keepalived won't help you with roaming at all - it works at a logical layer above the media layer, which is where roaming happens. I can sort of see why you might think you need it, if you're running devices which are unreliable but it would only be an Elastoplast and creates even more complexity. In the interests of the thread, at work I use keepalived very widely for HA failover of services in a world of virtual machines, so I can take say a webserver offline for maintenance but keep the site up on other VMs in the cluster with zero downtime.

Fast Transition is really intended for large installations with many tens or hundreds of access points, with hundreds to thousands (or more) clients. If your two access points are on different channels and in different places with a slight overlap, your connected devices should roam perfectly well.

Try to keep it simple.

1 Like