Policy-Based-Routing (pbr) package discussion

Hello,

Asking for a support!

I had a 23.05 based router, that had wireguard interface and all the traffic was going through wg0, except the rules I was setting with pbr.

Few days ago I changed the router and had to re-configure everything, I was taking previous configuration.

I have encountered a situation when nothing was routed outside of my network. And restart of pbr was helping to fix the routing.
I thought that is because of using 24.10rc and so re-installed with stable 23.05, but had same situation few more times.

But this time I was able to see the log before everything stopped working

Before pbr restart / When issue happens
Thu Jan  9 06:02:36 2025 daemon.info dnsmasq[1]: read /etc/hosts - 12 names
Thu Jan  9 06:02:36 2025 daemon.info dnsmasq[1]: read /tmp/hosts/dhcp.cfg01411c - 4 names
Thu Jan  9 06:02:36 2025 daemon.info dnsmasq[1]: read /tmp/hosts/odhcpd - 2 names
Thu Jan  9 06:02:36 2025 daemon.info dnsmasq-dhcp[1]: read /etc/ethers - 0 addresses
Thu Jan  9 06:02:36 2025 daemon.info dnsmasq[1]: read /etc/hosts - 12 names
Thu Jan  9 06:02:36 2025 daemon.info dnsmasq[1]: read /tmp/hosts/dhcp.cfg01411c - 4 names
Thu Jan  9 06:02:36 2025 daemon.info dnsmasq[1]: read /tmp/hosts/odhcpd - 4 names
Thu Jan  9 06:02:36 2025 daemon.info dnsmasq-dhcp[1]: read /etc/ethers - 0 addresses
Thu Jan  9 06:03:06 2025 daemon.notice netifd: wan (4047): udhcpc: sending renew to server 193.47.158.39
Thu Jan  9 06:03:06 2025 daemon.notice netifd: wan (4047): udhcpc: lease of 89.77.143.159 obtained from 193.47.158.39, lease time 172799
Thu Jan  9 06:03:06 2025 user.notice firewall: Reloading firewall due to ifupdate of wan (eth1)
Thu Jan  9 06:03:06 2025 user.notice pbr [32072]: Reloading routing for 'wan/eth1/89.77.142.1/::/0' [✓]
Thu Jan  9 06:03:06 2025 user.notice pbr: Sending reload signal to pbr due to firewall action: includes
Thu Jan  9 06:03:07 2025 user.notice pbr: Reusing the fw4 nft file.

Then I do restart of pbr and see in the log

After pbr restart
Thu Jan  9 06:08:06 2025 daemon.warn odhcpd[1894]: No default route present, overriding ra_lifetime!
Thu Jan  9 06:08:10 2025 user.notice pbr [728]: Removing routing for 'wan/eth1/89.77.142.1/::/0' [✓]
Thu Jan  9 06:08:10 2025 user.notice pbr [728]: Removing routing for 'wg0/10.65.106.171/fc00:bbbb:bbbb:bb01::2:6aaa/128' [✓]
Thu Jan  9 06:08:10 2025 user.notice pbr [728]: service (fw4 nft file mode) stopped [✓]
Thu Jan  9 06:08:10 2025 user.notice pbr [728]: Using wan interface (on_start): wan
Thu Jan  9 06:08:10 2025 user.notice pbr [728]: Found wan gateway (on_start): 89.77.142.1
Thu Jan  9 06:08:10 2025 user.notice pbr [728]: Using wan6 interface (on_start): wan6
Thu Jan  9 06:08:10 2025 user.notice pbr [728]: Setting up routing for 'wan/eth1/89.77.142.1/::/0' [✓]
Thu Jan  9 06:08:11 2025 user.notice pbr [728]: Setting up routing for 'wg0/10.65.106.171/fc00:bbbb:bbbb:bb01::2:6aaa/128' [✓]
Thu Jan  9 06:08:11 2025 user.notice pbr [728]: Routing 'direct-remote' via wan [✓]
Thu Jan  9 06:08:11 2025 user.notice pbr [728]: Routing 'direct-local' via wan [✓]
Thu Jan  9 06:08:11 2025 user.notice pbr [728]: Routing 'dns-google' via wan [✓]
Thu Jan  9 06:08:11 2025 user.notice pbr [728]: Routing 'dns-cloud' via wan [✓]
Thu Jan  9 06:08:11 2025 user.notice pbr [728]: Routing 'direct-4a' via wan [✓]
Thu Jan  9 06:08:11 2025 user.notice pbr [728]: Routing 'direct-4a-5g' via wan [✓]
Thu Jan  9 06:08:11 2025 user.notice pbr [728]: Routing 'direct-mipad' via wan [✓]
Thu Jan  9 06:08:11 2025 user.notice pbr [728]: Routing 'direct-citrix' via wan [✓]
Thu Jan  9 06:08:11 2025 daemon.warn odhcpd[1894]: No default route present, overriding ra_lifetime!
Thu Jan  9 06:08:11 2025 user.notice pbr [728]: Routing 'phone-4a' via wan [✓]
Thu Jan  9 06:08:11 2025 user.notice pbr [728]: Routing 'phone-4a-5g' via wan [✓]
Thu Jan  9 06:08:11 2025 user.notice pbr [728]: Routing 'k8-route' via wan [✓]
Thu Jan  9 06:08:12 2025 user.notice pbr [728]: Routing '845-eth-test' via wan [✓]
Thu Jan  9 06:08:12 2025 user.notice pbr [728]: Routing '845-wifi-test' via wan [✓]
Thu Jan  9 06:08:12 2025 user.notice pbr [728]: Installing fw4 nft file [✓]
Thu Jan  9 06:08:12 2025 user.notice pbr [728]: service monitoring interfaces: wan wg0
Thu Jan  9 06:08:12 2025 user.notice pbr [728]: service (fw4 nft file mode) started with gateways: wan/eth1/89.77.142.1/::/0 wg0/10.65.106.171/fc00:bbbb:bbbb:bb01::2:6aaa/128 [✓]
Thu Jan  9 06:08:13 2025 user.notice pbr: Sending reload signal to pbr due to firewall action: includes
Thu Jan  9 06:08:14 2025 user.notice pbr: Reusing the fw4 nft file.
Thu Jan  9 06:08:14 2025 user.notice SQM: Stopping SQM on eth1
Thu Jan  9 06:08:14 2025 user.notice SQM: Starting SQM script: piece_of_cake.qos on eth1, in: 450000 Kbps, out: 25000 Kbps
Thu Jan  9 06:08:15 2025 user.notice SQM: piece_of_cake.qos was started on eth1 successfully

Other times it happened I also see same lines in the log, like pbr is partly restarting but not fully. And on luci it says status is errored but without any information:

Thu Jan  9 06:03:06 2025 daemon.notice netifd: wan (4047): udhcpc: sending renew to server 193.47.158.39
Thu Jan  9 06:03:06 2025 daemon.notice netifd: wan (4047): udhcpc: lease of 89.77.143.159 obtained from 193.47.158.39, lease time 172799
Thu Jan  9 06:03:06 2025 user.notice firewall: Reloading firewall due to ifupdate of wan (eth1)
Thu Jan  9 06:03:06 2025 user.notice pbr [32072]: Reloading routing for 'wan/eth1/89.77.142.1/::/0' [✓]
Thu Jan  9 06:03:06 2025 user.notice pbr: Sending reload signal to pbr due to firewall action: includes
Thu Jan  9 06:03:07 2025 user.notice pbr: Reusing the fw4 nft file.

I don't know where to look for and what can cause it.
Any guidance?

Here is the configuration dump: https://pastebin.com/nqaDdY3A