Chiming in as a user of mwan3, but also help maintain the docs and keep an eye out on things.
The situation with mwan3 as I understand it currently is because it hasn't been fully converted to support nftables, so it relies on iptables compatibility related packages to allow for translation. Right now mwan3 essentially runs iptables through a nftables translation to generate valid firewall rules, rather than native.
The maintainer @feckert in fairness foresaw the ipset problem quite early on: Mwan3: nftables porting help - #3 by feckert. Under 22.03, the problem however wasn't as bad it looked as iptables-nft can help and dnsmasq supported ipset still in 22.03.
The issue now is the fact that the upstream dnsmasq-full package has now had it's default compile options updated to remove ipset support in favour of nftables in 23.05, given the firewall4 backend being nftables now established over two major releases. While this initially made some users annoyed because it's a major breaking change, the decision is based on the fact that only a few packages actually have any direct ipset dependencies now and the announcement of the iptables to nftables was made well in advance to prepare.
You can however appreciate that mwan3 being written originally for iptables and a firewall solution which OpenWrt has now dropped in core, now has a pretty major task of needing to be ported, but to ensure all the existing functionality works under the ported nftables version as well. Retesting it all for the variety of different network configurations and such is a major task.
For users of mwan3, that's all great, but what can users do? Here's some guides.
- Wait until mwan3 natively supports nftables.
- Use an alternative DNS resolver such as adguardhome, which has ipset support in a similar fashion to dnsmasq.
- Go back to the 22.03 release where dnsmasq-full has ipset support still
- Compile your own dnsmasq-full package with ipset enabled.
At the moment, for the least user inconvenience going back to 22.03 might be the easiest, 22.03 is still under active support and will be for a while.
As referenced there are tools to translate ipset to nftables but the issue will likely be the instant nature required of a DNS query returning one or more IP address being added into a set.
This is an important area however, the documentation needs to be updated to reflect the ipset issue, as it's a common feature used by many I'd imagine. I'll look at doing that.