Loopback is incontinent and looses its addresses

Installing and Using OpenWrt
1

After weeks of wondering what happens and why, and not coming up with an idea how to debug, I have now seen it by coincidence and can reproduce it.

What happens? My loopback devices looses all its IPv4 addresses.

My UCI interface config:

config interface            'loopback'
    option  device          'lo'
    option  proto           'static'
    list    ipaddr          '127.0.0.1/8'
    list    ipaddr          '192.168.0.1/32'
    list    ipaddr          'fde6:a09a:b373::1/128'

So normally it looks like

lo               UNKNOWN        127.0.0.1/8 192.168.0.1/32 fde6:a09a:b373::1/128 ::1/128

And now when my wireguard interface restarts, the IPv4 addresses are gone:

root@cpe:~# date; ip -br addr show dev lo
Sun Dec 11 22:00:50 UTC 2022
lo               UNKNOWN        127.0.0.1/8 192.168.0.1/32 fde6:a09a:b373::1/128 ::1/128

root@cpe:~# date; ./restart-wg.sh
Sun Dec 11 22:01:36 UTC 2022

root@cpe:~# date; ip -br addr show dev lo
Sun Dec 11 22:01:42 UTC 2022
lo               UNKNOWN        fde6:a09a:b373::1/128 ::1/128

root@cpe:~# date; logread -f | grep -vEe 'dnsmasq-dhcp|cron|Destination address required'
Sun Dec 11 22:01:28 UTC 2022
Sun Dec 11 22:01:36 2022 daemon.notice netifd: Network device 'wg0' link is down
Sun Dec 11 22:01:36 2022 daemon.warn bird: Netlink: No such device
Sun Dec 11 22:01:36 2022 daemon.warn bird: Netlink: No such device
Sun Dec 11 22:01:36 2022 daemon.warn bird: Netlink: No such device
Sun Dec 11 22:01:36 2022 daemon.notice netifd: Interface 'wg0' is now down
Sun Dec 11 22:01:36 2022 daemon.warn bird: Netlink: No such device
Sun Dec 11 22:01:36 2022 daemon.warn bird: Netlink: No such device
Sun Dec 11 22:01:36 2022 daemon.warn bird: ...
Sun Dec 11 22:01:36 2022 daemon.err bird: KRT: Received route 2003:e4:xxxx:xxxx::/64 with unknown ifindex 47
Sun Dec 11 22:01:36 2022 daemon.err bird: KRT: Received route fde6:a09a:b373:f0::/64 with unknown ifindex 47
Sun Dec 11 22:01:36 2022 daemon.err bird: KIF: Received address message for unknown interface 47
Sun Dec 11 22:01:36 2022 daemon.err bird: KIF: Received address message for unknown interface 47
Sun Dec 11 22:01:36 2022 daemon.err bird: KIF: Received address message for unknown interface 47
Sun Dec 11 22:01:36 2022 daemon.err bird: KIF: Received address message for unknown interface 47
Sun Dec 11 22:01:37 2022 daemon.notice netifd: Interface 'wg15' is now down
Sun Dec 11 22:01:38 2022 daemon.notice netifd: Interface 'loopback' is setting up now
Sun Dec 11 22:01:38 2022 daemon.notice netifd: Network device 'wg15' link is down
Sun Dec 11 22:01:38 2022 daemon.notice netifd: Interface 'wg0' is setting up now
Sun Dec 11 22:01:38 2022 daemon.notice netifd: Interface 'loopback' is setting up now
Sun Dec 11 22:01:39 2022 daemon.notice netifd: Interface 'wg15' is setting up now
Sun Dec 11 22:01:41 2022 daemon.err bird: ospf6: Socket error on wg0: Invalid argument
Sun Dec 11 22:01:41 2022 daemon.err bird: ospf4: Socket error on wg0: Invalid argument
Sun Dec 11 22:01:41 2022 daemon.notice netifd: Interface 'wg0' is now up
Sun Dec 11 22:01:41 2022 daemon.notice netifd: Network device 'wg0' link is up
Sun Dec 11 22:01:41 2022 daemon.notice netifd: Interface 'wg15' is now up
Sun Dec 11 22:01:41 2022 daemon.notice netifd: Network device 'wg15' link is up
Sun Dec 11 22:01:42 2022 user.notice firewall: Reloading firewall due to ifup of wg0 (wg0)
Sun Dec 11 22:01:44 2022 user.notice firewall: Reloading firewall due to ifup of wg15 (wg15)

The only thing I can spot here is:

Sun Dec 11 22:01:38 2022 daemon.notice netifd: Interface 'loopback' is setting up now

and again

Sun Dec 11 22:01:38 2022 daemon.notice netifd: Interface 'loopback' is setting up now

I assume because I have two wireguard interfaces and something is triggered two times?
But besides that? I have no clue what netifd does here with the loopback and why it do so.

I'm still on OpenWrt 21.02.1 r16325-88151b8303.

2

This is wrong. You should not have additional IP addresses in the loopback.
It should look like this:

config interface 'loopback'
        option device 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'
2 Likes
3

Not its not. https://openwrt.org/docs/guide-user/network/ipv4/configuration#multiple_ip_addresses

However, I spotted another typo

    list    ipaddr          'fde6:a09a:b373::1/128'

should be

    list    ip6addr         'fde6:a09a:b373::1/128'

But still. What is doing netifd with loopback when I restart my wireguard interfaces?

By the way, fixing that typo solved my issue.

Edit:

You should not have additional IP addresses in the loopback.

I'm not joking when I now tell you that nearly everyone involved in networking has more then the loopback IP 127.0.0.1/8 and ::1 assigned to his/her/* loopback interface. Because normally you will never loose loopback aka it should never go down. So your addresses on the loopback interfaces will never be down.

4

That would be easier to answer if you would disclose the contents of your ./restart-wg.sh script.

Clearly the runtime state of your loopback interface (has IPv6 address) deviates from the configuration state (does not contain IPv6 address, or rather has semantic error). So something besides netifd must've set it up initially. Eventually when netifd reloads it's configuration it will try to synchronize, borking the loopback l3 setup.

5

Content is only ifdown, sleep, ifup.
I think it was because of the sync you mentioned and my typo that I assigned an IPv6 address to ipaddr.
But this seams like a funny bug that this does not throw an error and that in the end only this IPv6 address was assigned....

6

I've been working it IT industry for +25 years, and this is the first time I read about such configuration...

7

Agree, there is no reason to put external IPs on loopback, and in fact nobody expects anyone to do that, so you "unlock bugs" doing so.

1 Like
8

Please read how enterprise and ISP build their core networks. The classical one is running Ospf to redistribute your loopback addresses and iBGP is using these for the peering config. Really both of you making me speechless.
Any serious anycast config operates in a similar level.

Ps: take for instance this brief intro: https://packetpushers.net/understanding-ospf-router-id-rid-assignment/

9

I have to agree with @_bernd here. Binding additional IP addresses (e.g. "floating" kinds such as VRRP / failover ones) to loopback is pretty common practice in advanced network setups.

My hunch is that the IPv6 address was simply not being removed yet by netifd. Since it was incorrectly specified in the config, netifd wouldn't have had any knowledge about it in its runtime state.

It could probably be more verbose about such errors, though.

1 Like
10

Just my 2 cents here.
It is quite common to use loopback interfaces to have addresses independent of the interface state, so @_bernd is right here.
What I haven't seen before is to overload the lo interface with such addresses. What we usually do is create additional interfaces, like loopback1 or dummyX, so I am not sure I would agree with this approach.

11

I assume that the practice of using additional loopback interfaces comes from using Cisco where only a single address could be assigned if my memory serve me well. Having more then one IPv4 address on any interface on Linux however is supported since ages....

12

Wrong assumption.

Usage Guidelines

An interface can have one primary IP address and multiple secondary IP addresses.

13

Yes I know secondary interfaces addresses but I wanted to say back back in the days.... When IPv4 got mainstream it was never planed that an interface should have more then one address and Linux got criticism when it introduced the behavior of answer Arp on an interface even the requested address was assigned to another address..... But I know of no good reason why more then one loopback would be needed nowadays.

14

I guess a combination of anycast, ospf loopback, bgp loopback, or one loopback per server in a multi-interface server. But that would be a very particular case.

15

Ok yes if you need more then one Ospf protocol instance then a second or third loopback make sense. But for bgp peering and anycast I would just throw everything at loopback. Doing this since years and have not seen yet any wonky behaviour :man_shrugging:

16

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.