Dnsmasq nftset/ipset to local domain

[xize]

Hello,

So I defined some domain names in /etc/dnsmasq.conf for steam's lancache server.

But when I want to use my wgserver I do not want to reach these dns domains, so I thought of using a ipset/nftset through dnsmasq to make more selective routing for my dns.

My idea is to have two dns hijack rules:

1. for my local services like jellyfin.lan I want to hold a ipset and use hijack rules towards the routers dns.
2. for everything else I want to hijack dns to 0.0.0.0:5342 (luci-app-nextdns) so I can still use nextdns and not deal with dnsmasq the ipset rule should handle that as higher priority that is my idea.

When I create a ipset through luci via the firewall options, and specify dest_ip this should be fine.(?)

When I create the ipset inside the dhcp options in luci and then go inside the CLI to remove the option table_family then it must work correctly because for my other ipset for tmdb this was required.

Now my issue is this:

I created the exact same config for tmdb.org and I see this ipset being populated in nft list sets.

But it doesn't do that on my local domains why?, is this a limitation with nft and dnsmasq?

this is how I configurated them in:

firewall:

config ipset
        option name 'tmdb'
        option family 'ipv4'
        list match 'dest_ip'
        option counters '1'

config ipset
        option name 'wgserver-local'
        option comment 'force wgserver to resolve jellyfin instances'
        option family 'ipv4'
        option counters '1'
        list match 'dest_ip'

dhcp (I copied only the needed bits because my lab network is huge):

config dnsmasq
        option rebind_protection '1'
        option local '/lan/'
        option domain 'lan'
        option expandhosts '1'
        option cachesize '0'
        option authoritative '1'
        option readethers '1'
        option leasefile '/tmp/dhcp.leases'
        option localservice '1'
        option ednspacket_max '1232'
        option sequential_ip '1'
        option domainneeded '1'
        option dnsseccheckunsigned '0'
        option filter_aaaa '1'
        option rebind_localhost '1'
        option localise_queries '1'
        option noresolv '1'

config ipset
        list name 'tmdb'
        list domain 'image.tmdb.org'
        list domain 'themoviedb.org'
        list domain 'tmdb.org'

config ipset
        list name 'wgserver-local'
        list domain 'jellyfin.lan'
        list domain 'jellyseerr.lan'
        list domain 'sonarr.lan'
        list domain 'radarr.lan'
        list domain 'prowlarr.lan'
        list domain 'bazarr.lan'

and the nft list sets command:

table inet fw4 {
        set tmdb {
                type ipv4_addr
                elements = { 18.239.xx, xxx }
        }
        set wgserver-local {
                type ipv4_addr
                comment "force wgserver to resolve jellyfin instances"
        }
        set pbr_wan_4_dst_ip_cfg036ff5 {
                type ipv4_addr
                flags interval
                auto-merge
                comment "bypass domain"
                elements = { 2.1x.x.x, xxx }
        }
}

I already tried setting dnsmasq cache to 0, restarted the router, flushed my dns with nslookup but it won't populate, I thereof think it must be a limitation?

Thanks!

edit

I don't think the dns stuff will work since the client device only sents a dest after resolved so the nft set never get filled, but I'm still happy to learn why local domains won't get ipsetted by dnsmasq since I'm not hijacking, just checking if it fills the ip set , it seem to be exclusively to remote domains.

[brada4]

dnsmasq.conf is generated at start from config/dhcp.
check ps auxw for the real file

[dave14305]

It looks like dnsmasq only populates nft sets for domain queries that get forwarded upstream. Queries answered locally do not seem to trigger the add_to_nftset() function. I suppose if the local domain is managed locally, you would already know the local IPs or MAC addresses to populate in a set?

[xize]

@dave14305

you would already know the local IPs or MAC addresses to populate in a set?

yup that is correct.

@dave14305

It looks like dnsmasq only populates nft sets for domain queries that get forwarded upstream. Queries answered locally do not seem to trigger the add_to_nftset() function.

thanks for verifying!, I was not sure about this.

--

well I don't think hijacking is the solution for this even if I would use a hardcoded ipset, I need to figure something else I could split route on the wireguard client side that might be a better idea, atleast I know now dnsmasq ipsets are only for upstream domains now