something might not be right as per following output when attempting to execute:
hotplug-call online
/sbin/hotplug-call: /etc/hotplug.d/online/60-ipset-doh: line 18: syntax error: unexpected "(" (expecting "}")
Can't lock /var/lock/doh.lock
uci: Entry not found
Usage: lock [-suw] <filename>
-s Use shared locking
-u Unlock
-w Wait for the lock to become free, don't acquire lock
-n Don't wait for the lock to become free. Fail with exit code
did you create the doh ipset? that is prerequisite.
this is just an old shell trick, my wan is also called "wan" but if $INTERFACE was blank than testing of condition would fail: comparing nothing with "wan" is not possible. but comparing - worst case - "x" with "xwan" is possible. in ideal case x$INTERFACE evaluates to "xwan" so matching against "xwan" would be ok.
am I right that it is created by 60-ipset-doh shown on DNS hijacking and is triggered when interface goes online ?
I have that but ipset is not being populated.
Manual trigger via
ipset setup
does not do much either.
I must be missing something but no idea where to start looking.
I updated the code in the wiki to optimize dependencies and use already resolved blocklists.
Resolving DoH servers locally one by one is too time consuming since their number has significantly increased.
The dibdot/DoH-IP-blocklists repo provides daily updates for the resolved blocklists and basically solves the problem.
what about start reading this thread from the beginning
detailed answer in previous posts, short answer: in openwrt 22.x ipset is not working with nftables so you should not rely on ipset extra script (which includes ipset setup).
use luci network / dhcp / ip sets and create two (empty) sets called doh and doh6. then you use the script above.
after @vgaetera kindly solved the heavy lifting to resolve public DoH addresses (see his post) [ really appreciate ] it became much easier to add those address to nft set.
just add configuration to /etc/config/firewall:
config ipset doh
option name doh
option family ipv4
option match 'dest_ip'
option loadfile '/tmp/doh4.txt'
config ipset doh6
option name doh6
option family ipv6
option match 'dest_ip'
option loadfile '/tmp/doh6.txt'
and create the below script in /etc/hotplug.d/online directory:
Except that the big DoH providers seem to roll through a collection of IP addresses, so dibodot lists only give you a snapshot that blocks the ones found in the instant they were created...
$ nft list sets
table inet fw4 {
set doh_ipv4 {
typeof ip daddr
timeout 7d
gc-interval 6h
comment "EF-DNS: Block list for IPv4 DoH hosts."
}
set doh_ipv6 {
typeof ip6 daddr
timeout 7d
gc-interval 6h
comment "EF-DNS: Block list for IPv6 DoH hosts."
}
}
And then I update the sets, rather than replacing them wholesale, in hopes of accumulating the transients.
Complicating this is the fact that there's no out-of-band method for updating set elements using nft (my post on netfilter dev list: https://marc.info/?l=netfilter&m=166525399226074&w=2), so I have resorted to creating a dummy rule, firing off that rule for each IP to update the element, then delete the rule. Blarg, it's not exactly what I'd call "elegant", but it seems to work. (Oh, and I also supplement it with a DoH host list in my DNS blocker.)
I'm afraid that DNS-to-IP-set based filtering has some loopholes in its design.
If you get a different reply every N minutes, the actual TTL could be even smaller, and you need to update even more often, perhaps once a minute, since there's always a chance that you do not update often enough.
However, if you update too often, it will result in a flood of meaningless DNS traffic as most domains likely do not need frequent updates.
On the other hand, the reply may be unrelated to TTL and instead just return a limited set of addresses from the pool on each N-th query, so it would require to block entire subnets probably by ASN.
In this case, populating IP sets dynamically with dnsmasq looks like a more reasonable approach.
Another major issue is resilience to reboots and power outages, so the faster you populate IP sets at startup, the sooner your setup becomes ready, and in this context there's no point to rely on runtime configuration that can be reset at any moment.
It could be a different story on a general purpose system with persistent data storage not affected by flash wearing, but OpenWrt has its own specifics that we need to consider.
Yeah, without some sort of DPI (deep packet inspection) on the actual https traffic, I think there's always going to be some holes that people need to be aware of. Maybe we should put a warning in the wiki "DoH blocking works pretty well, but beware of these issues..."
(Hmm, I wonder if snort has enough smarts to do this sort of DPI detection??? I'm not familiar enough with it to say yes, no or maybe.)
My own simple experiments with applying Dirk's dibodot IP lists in firewall rules, and his domain name list with dnsmasq/adblock shows that some things still slip by the host list and get caught by the IP list (I have no idea, of course, what gets by both!).
I log when something gets past DNS to the firewall, and right now see these three IPs that got flagged in the past couple days:
I haven't yet instrumented things enough to know the specific domain names used, but the first one is some sort of AWS instance (dig -x says a51062ecadbb5a26e.awsglobalaccelerator.com), and the other two are something on bunnycdn.com registered in Slovenia.
I simply add the relevant domains (mainly apple) to the local banIP blocklist...and over time I collect/block all relevant IPs, cause banIP resolves domain names with every run.
raises the question of effort cost vs expected result. you said @efahl that big providers refreshing their ip pool continuously, indeed it is true, so updating every 6hours is also not 100% accurate, but imho not that much value add (e.g. cloudflare is providing many/every thing via same ip pool and port 443 not just DoH, blocking would be contra-productive).
