OpenWrt-Azure VPN with strongswan + BGP with Bird - received route with strange next-hop

Hello,
my OpenWrt router has a site-to-site VPN with Azure made with strongswan. The Azure VPN gateway has two active/active instances, that OpenWrt reaches using two xfrm interfaces, 'xfrm0' and 'xfrm1'. The gateway also supports BGP and has the following BGP peers on Azure: 10.0.3.196 (used by the first VPN gateway instance) and 10.0.3.197 (the second).

In order to reach the two peers over the VPN, I have the following routes in /etc/config/network:

config route
        option interface 'xfrm0'
        option target '10.0.3.196/32'
        option source '192.168.1.1'

config route
        option interface 'xfrm1'
        option target '10.0.3.197/32'
        option source '192.168.1.1'

Everything works and OpenWrt receives and uses the announced routes from the two peers, but the log is filled with messages from Bird like 'received route with strage next-hop 10.0.3.196' (or .197).

Any hint to improve the configuration and avoid this? In /etc/bird.conf I have something like:

protocol bgp bgp1 {
    local 192.168.1.1 as 64512;
    neighbor 10.0.3.196 as 65515;
    multihop;
    debug all;

    ipv4 {
        import filter {
                 print "BGP1: IMPORT ", net, " gw ", gw, " bgp_next_hop ", bgp_next_hop;
                 accept;
         };
         export filter {
                  print "BGP1: EXPORT ", net, " gw ", gw, " bgp_next_hop ", bgp_next_hop;
                  if net = 0.0.0.0/0 then reject;       # don't send 0.0.0.0/0 to our neighbors
                  else accept;
         };
    };
}

Thank you,
Marcello

Could you share the full bird config?

here it is, thank you:

log syslog all;
debug protocols all

define SOURCE_IP = 192.168.1.1;

router id 192.168.1.1;

# The Device protocol is not a real routing protocol. It doesn't generate any
# routes and it only serves as a module for getting information about network
# interfaces from the kernel.
protocol device {
        scan time 60;
}

# the kernel protocol is used to exchange routes with kernel:  BGP <--> KERNEL
protocol kernel {               # using primary routing table
        learn all;              # Learn alien routes from the kernel
        merge paths on;         # Usually, only best routes are exported to the kernel protocol. With path merging enabled, both best routes and equivalent non-best routes are merged during export to generate one ECMP (equal-cost multipath) route for each network
        scan time 10;           # Scan kernel routing table every 10 seconds
        ipv4 {
                import filter {
                  print "KERNEL: IMPORT ", net, " gw ", gw, " bgp_next_hop ", bgp_next_hop;
                  accept;
          };
          export filter {

                if source = RTS_INHERIT then accept;    # routes inherited from other protocols

                if source = RTS_BGP then
                {
                        krt_realm = bgp_path.first;
                        krt_prefsrc = SOURCE_IP;
                        print "KERNEL: EXPORT ", net, " gw ", gw, " bgp_next_hop ", bgp_next_hop;
                        accept;
                }

                reject;
          };
     };
}

protocol bgp bgp1 {
    local 192.168.1.1 as 64512;
    neighbor 10.0.3.196 as 65515;
    multihop;
    debug all;

    ipv4 {
        import filter {
                 print "BGP1: IMPORT ", net, " gw ", gw, " bgp_next_hop ", bgp_next_hop;
                 accept;
         };
         export filter {
                  print "BGP1: EXPORT ", net, " gw ", gw, " bgp_next_hop ", bgp_next_hop;
                  if net = 0.0.0.0/0 then reject;       # don't send 0.0.0.0/0 to our neighbors
                  else accept;
         };
    };
}

protocol bgp bgp2 {
    local 192.168.1.1 as 64512;
    neighbor 10.0.3.197 as 65515;
    multihop;
    debug all;

    ipv4 {
        import filter {
                print "BGP2: IMPORT ", net, " gw ", gw, " bgp_next_hop ", bgp_next_hop;
                accept;
        };
        export filter {
                print "BGP2: EXPORT ", net, " gw ", gw, " bgp_next_hop ", bgp_next_hop;
                if net = 0.0.0.0/0 then reject;       # don't send 0.0.0.0/0 to our neighbors
                else accept;
        };
    };
}

How does the kernel routing table look for these routes (the peer address)?
I see you add the peer host route with uci so it should be in the kernel table already...
Maybe it helps to add a protocol static stanza?

protocol static {
        ipv4;
        check link;
        route 10.0.3.196/32 dev "xfrm0" onlink;
        route 10.0.3.197/32 dev "xfrm1" onlink;
}

(I'm not 100% sure, but I assume/think you need to set the route onlink. Either as a flag using ip from iproute2, or as a flag within the protocol static stanza....

Thank you,
everything actually works without warnings if I remove the routes from /etc/config/network, don't use any 'protocol static' section in bird.conf, and instead manually execute:

ip route add 10.0.3.196/32 via 10.0.3.196 dev xfrm0 onlink
ip route add 10.0.3.197/32 via 10.0.3.197 dev xfrm1 onlink

However, I cannot find a way to put these routes either in the /etc/config/network file, or in a 'protocol static' section in bird.conf, so that they are automatically configured at reboot.

If I try your suggestion, Bird tries to install the routes but it gets:

daemon.warn bird: Netlink: Invalid argument

Would it help to add them in /etc/rc.local or some hotplug script to come up on boot or when an interface comes up?

Please try

protocol static "static1" {
        ipv4;
        check link;
        route 10.0.3.196/32 dev "xfrm0" onlink;
        route 10.0.3.197/32 dev "xfrm1" onlink;
}

and if birdc configure throws an error, please post that...
I would recommend to have these routes within bird.conf.

Ok, I had to adjust the code slightly, because 'static1' and 'dev' were not accepted:

BIRD 2.14 ready.
Reading configuration from /etc/bird.conf
/etc/bird.conf:9:25 syntax error, unexpected TEXT
BIRD 2.14 ready.
Reading configuration from /etc/bird.conf
/etc/bird.conf:12:29 syntax error, unexpected DEV

Looking at the syntax in the documentation (https://bird.network.cz/?get_doc&v=20&f=bird.html#toc6.17), I tried:

protocol static {
        ipv4;
        check link;
        route 10.0.3.196/32 via "xfrm0" onlink;
        route 10.0.3.197/32 via "xfrm1" onlink;
}

birdc configure is ok with that:

BIRD 2.14 ready.
Reading configuration from /etc/bird.conf
Reconfigured

However, logread shows:

Wed Apr  3 18:12:06 2024 daemon.debug bird: kernel1: 10.0.3.197/32: installing
Wed Apr  3 18:12:06 2024 daemon.warn bird: Netlink: Invalid argument
Wed Apr  3 18:12:06 2024 daemon.debug bird: kernel1: 10.0.3.196/32: installing
Wed Apr  3 18:12:06 2024 daemon.warn bird: Netlink: Invalid argument

The routes are not listed in the output of ip route...

Pardon me. (I think there is an issue with the docs.)
Try this please:

protocol static {
        ipv4;
        check link;
        route 10.0.3.196/32 via "xfrm0";
        route 10.0.3.197/32 via "xfrm1";
}

(
Tested with route 203.0.113.1/32 via "lo"; and

# ip route show 203.0.113.1
203.0.113.1 dev lo proto bird scope link metric 32

# birdc show route for 203.0.113.1
BIRD 2.0.8 ready.
Table master4:
203.0.113.1/32       unicast [static_ipv4 16:58:13.962] * (200)
        dev lo

)

Hi, with this block:

protocol static {
ipv4;
check link;
route 10.0.3.196/32 via "xfrm0";
route 10.0.3.197/32 via "xfrm1";
}

it works, but...

root@OpenWrt:/etc# ip route show 10.0.3.196
10.0.3.196 dev xfrm0 proto bird scope link metric 32
root@OpenWrt:/etc# birdc show route for 10.0.3.196
BIRD 2.14 ready.
Table master4:
10.0.3.196/32        unicast [static1 20:51:11.017] * (200)
        dev xfrm0
root@OpenWrt:/etc# birdc show route for 10.0.3.197
BIRD 2.14 ready.
Table master4:
10.0.3.197/32        unicast [static1 20:51:11.017] * (200)
        dev xfrm1
root@OpenWrt:/etc# ip route show 10.0.3.197
10.0.3.197 dev xfrm1 proto bird scope link metric 32

...but I have the continuous warnings that fill the log, one for each prefix received from BGP:

daemon.err bird: KRT: Received route A.B.C.D/N with strange next-hop 10.0.3.196

:frowning_face:

I assume you have searched the web already too, could post your issue on the mailing list please https://bird.network.cz/mailman/listinfo/bird-users ?

Have you tried to delete this?

You can also try to set interface explicit within the protocol bgp stanza.

Thank you,
I have tried also the last suggestions, but there were no changes.

Looking at Bird's mailing list, I have found KRT: Received route with strange next-hop (network.cz).

I have tried the suggestion in that mail and changed approach: I have removed the static protocol block from bird.conf, and created a hotplug script for xfrm0/xfrm1, that:

  • adds a local address and remote address = 10.0.3.196 to xfrm0, if action is ifup for xfrm0
  • adds a local address and remote address = 10.0.3.197 to xfrm1, if action is ifup for xfrm1
  • removes the addresses above for xfrm0/xfrm1 when their action is ifdown.

Using hotplug scripts was also one of the other suggestions in this thread. I hoped to avoid scripts and use just configuration files, but this solution seems to work very well: not only I don't have the warnings anymore, but the log of Bird looks much cleaner now, i.e.. other minor warnings that I had before have disappeared.

Thank you for your support!

2 Likes

Please mark your last post as "solved" for this thread.

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.