Need help with Wireguard and BIRD/OSPF

tyami94 · January 11, 2025, 5:08pm

Hi Everyone,

I'm having some trouble getting dynamic routing working over a Wireguard tunnel. No matter what I do, the devices will not see each other as OSPF neighbors. tcpdump shows no OSPF traffic of any kind on either router. Alongside OSPF, I have BIRD configured to add a static route to the Wireguard tunnel as a bootstrap, and this does work. The Wireguard tunnel is up and I can pass traffic through it as expected using the route added by BIRD. I can't figure out where the problem is as I'm not familiar enough with OSPF, BIRD, and nftables to properly debug it (which is why I am attempting this, hoping I'll learn more about these things).

Before I attempted this, I got BIRD set up and working between 2 laptops running Alpine with an almost identical config (including Wireguard) and it works, but I cannot reproduce this behavior under OpenWRT. Only major difference is nftables was not enabled on the laptops. Hoping I just made a dumb mistake somewhere and just need another set of eyes. Update: This is wrong, and I am dumb.

Thanks in advance,
--tyami94

Below are snippets of the relevant config files. Both devices are set up near identically.

Router 1

/etc/bird.conf

router id 10.0.0.1;
debug protocols all;

protocol device {
}

#Initial route to bootstrap OSPF
protocol static bootstrap {
    ipv6;
    route fde0:690a:be3e::/48 via "sitetosite";
}

protocol direct self {
    ipv6;
    interface "sitetosite";
}

protocol kernel {
    ipv6 {
        export filter {
            if proto = "self" then reject; #Do not add our own IP to the FIB
            accept;
        };
    };
}

protocol ospf v3 ospf1 {
    ipv6 {
        export all;
    };
    area 0 {
        interface "sitetosite" {
            type ptmp;
            neighbors {
                fde0:690a:be3e::1;
            };
        };
    };
}

/etc/config/firewall (partial)

config defaults
        option input 'REJECT'
        option output 'ACCEPT'
        option forward 'REJECT'
        option synflood_protect '1'

---SNIPPED---

config zone
        option name 'sitetosite'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'ACCEPT'
        list network 'sitetosite'
        option log '1'
        option log_limit '10/minute'

config rule
        option name 'Allow-Wireguard-Site-to-Site-In'
        list proto 'udp'
        option src '*'
        option dest_port '51821'
        option target 'ACCEPT'

config rule
        option name 'Allow-OSPF-In'
        list proto 'ospf'
        option src 'sitetosite'
        option target 'ACCEPT'
        option family 'ipv6'

config rule
        option name 'Allow-OSPF-Out'
        list proto 'ospf'
        option dest 'sitetosite'
        option target 'ACCEPT'
        option family 'ipv6'

/etc/config/network (partial)

config interface 'sitetosite'
        option proto 'wireguard'
        option private_key '<Redacted>'
        option listen_port '51821'
        list addresses 'fde0:690a:be3e::'
        option defaultroute '0'
        option delegate '0'
        option nohostroute '1'

config wireguard_sitetosite
        option description 'Router2'
        option public_key '<Redacted>'
        option preshared_key '<Redacted>'
        option endpoint_host '<Redacted>'
        option endpoint_port '51821'
        option persistent_keepalive '25'
        list allowed_ips '::/0'
        list allowed_ips '0.0.0.0/0'

Router 2

/etc/bird.conf

router id 10.2.0.1;
debug protocols all;

protocol device {
}

#Initial route to bootstrap OSPF
protocol static bootstrap {
    ipv6;
    route fde0:690a:be3e::/48 via "sitetosite";
}

protocol direct self {
    ipv6;
    interface "sitetosite";
}

protocol kernel {
    ipv6 {
        export filter {
            if proto = "self" then reject; #Do not add our own IP to the FIB
            accept;
        };
    };
}

protocol ospf v3 ospf1 {
    ipv6 {
        export all;
    };
    area 0 {
        interface "sitetosite" {
            type ptmp;
            neighbors {
                fde0:690a:be3e::;
            };
        };
    };
}

/etc/config/firewall (partial)

config defaults
        option input 'REJECT'
        option output 'ACCEPT'
        option forward 'REJECT'
        option synflood_protect '1'

config zone
        option name 'sitetosite'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'ACCEPT'
        list network 'sitetosite'

config rule
        option name 'Allow-Wireguard-Site-to-Site-In'
        list proto 'udp'
        option src '*'
        option dest_port '51821'
        option target 'ACCEPT'

config rule
        option name 'Allow-OSPF-In'
        list proto 'ospf'
        option src 'sitetosite'
        option target 'ACCEPT'
        option family 'ipv6'

config rule
        option name 'Allow-OSPF-Out'
        list proto 'ospf'
        option dest 'sitetosite'
        option target 'ACCEPT'
        option family 'ipv6'

/etc/config/network (partial)

config interface 'sitetosite'
        option proto 'wireguard'
        option private_key '<Redacted>'
        option listen_port '51821'
        list addresses 'fde0:690a:be3e::1'
        option defaultroute '0'
        option delegate '0'
        option nohostroute '1'

config wireguard_sitetosite
        option description 'Router1'
        option public_key '<Redacted>'
        option preshared_key '<Redacted>'
        option endpoint_host '<Redacted>'
        option endpoint_port '51821'
        option persistent_keepalive '25'
        list allowed_ips '::/0'
        list allowed_ips '0.0.0.0/0'

tyami94 · January 11, 2025, 7:53pm

Figured it out. BIRD/OSPFv3 relies on IPv6 link-local addresses. Since Wireguard tells the kernel not to assign link-local addresses by default, both instances just sit there doing nothing. BIRD does not output any log messages to tell you that it can't operate without a link-local address, so I was just kind of stuck. Not sure if this is best-practice, but I just used fe88::/64 for my interface IPs instead of the ULA's as before, and things magically started to work.

And to address the mystery of why the laptops worked but OpenWRT didn't, turns out they didn't work. I had the laptops set up to use OSPFv2/IPv4 instead of OSPFv3/v6. I thought the difference would be minimal (which is why I said the config was near identical), but I guess I was wrong lol. Turns out there's a more of a difference then I thought. My fault.

Found the fix here: https://idndx.com/ospf-over-wireguard/

Hope this is helpful to someone,
Thanks

_bernd · January 11, 2025, 8:03pm

Some add-ons,

Set explicit the multicast flag on the wg interfaces. Openwrt can do this via network config, ifupdown on Debian needs a post up script.
Just to be not that wired but link local is fe80. Please adjust just get a random Mac address, generate the lla and configure it statically.
If you want to import addresses from loopback you have to add an lla to loopback too. This is funny kind of bug kind of feature with bird2.
Happy routing!

Edit bonus
If you have point to multipoint wg connections you have to set this type on the interface config within bird for ospf.

Edit2
For site to site tunnel the type point-to-point in the bird interface config does not hurt too

tyami94 · January 11, 2025, 10:12pm

This is extremely helpful, as soon as I added another peer and did point-to-multipoint everything fell apart, presumably for all the reasons you've mentioned here. Very new to IPv6 as well, just different enough that I almost understand it, but not to the same degree as v4, def should've known better about fe88:: being invalid though lol. Trying your advice now, thanks!

_bernd · January 11, 2025, 11:18pm

There is even more: dark magic / trickery to do something just like "IPv4 Unnumbered".
Normally that would mean to have no IPv4 address on an interface at all, but to be able to have a source address, you "borrow" i.e. an address from loopback.
Linux however can't do this, .... but what Linux can do, is: You "recycle" your Router ID -- hopefully assigned as /32 to loopback -- and use it on all your other interfaces.

tyami94 · January 12, 2025, 5:26pm

So I've hit a wall here, when I add another peer to the existing tunnel, the system falls apart. It seems that this is because wireguard does not know where to route my traffic if both peers AllowedIPs = 0.0.0.0/0, ::/0. I figured there would be a way to control which peer traffic is forwarded to in the kernel routing table, but that doesn't seem to be the case. Do you know of a workaround for this that doesn't require me to set up multiple tunnels, or is that the best one can do currently?

I found this on the lkml, and it seems to be discussing pretty much the exact problem I'm running into, but I didn't see any obvious workarounds (at least ones that i understand): https://lore.kernel.org/all/20230819140218.5algu2nfmfostngh@House.clients.dxld.at/T/

_bernd · January 12, 2025, 5:34pm

Either you restrict the allowed IP to cover the routes which will be announced or you have to move each connection to a dedicated peer to peer setup. One wg interface per peer. And no, do not enable add allowed ips or however that setting was called.

Safest option is one interface per peer. This gave me the least headache.