Not my work, just reposting here for more people to see. I'm using this method - it's truly insanely fast, and the full OISD blocklist is only consuming ~30mb ram on my r7800 / 22.03.
Thanks to luigi_xp whoever/wherever you are. Link to original reddit thread:
22.03-rcx comes with dnsmasq 2.86, which brings a full rewrite of the dns handling code making it much more efficient in both ram and processing for various things, including common blocklist syntax.
I'm running the full 1M+ line oisd blocklist with no noticeable performance penalty on my humble MT7621/128MB Archer C6.
I created a script which is literally
wget https://dnsmasq.oisd.nl -O /tmp/dnsmasq.d/oisd.txt && /etc/init.d/dnsmasq restart
and setup it to run every day and on startup, and it's working silk smooth. Dnsmasq is using a "large" amount of ram (17%), but considering what it's doing and that there's still a lot of free ram left, i'm happy with the result.
Unfortunately plugins such as adblock still choke because they process the lists before, but if you use dnsmasq directly and create a little script to download and update the lists it works as good if not better than the standalone plugins.
Hope someone finds this useful
Great - I run dnsmasq, now also at version 2.86, on my Linux desktop, with several blocklists that in total, in their human readable text format, consume over 27 Mbytes. I've been using dnsmasq since at least 2016. It's outstanding.
Kudos for sharing this, afaik there are lot of people here on the forum who don't frequent reddit and vice a versa.
Obviously to each their own, and yes, that blocklist would probably choke
simple-adblock on a 128Mb router, but there are upsides of "processing", especially if you're using multiple sources --
adblock pioneered and
simple-adblock shamelessly copied the functionality to remove duplicates and unnecessary 3rd level domains from the final block list for supported dnsmasq/unbound configs.
Another upside of using
simple-adblock is a built-in enforcement of OpenWrt's resolver across your network, so even if your clients are set up to use hardcoded DNS (but not DoT/DoH) servers, the ads would still be blocked.
PS. Have you run any actual tests to measure the performance of dnsmasq with and without that list?
Hey no problems re-posting it here. Exactly why I did it, I think a lot of people would never visit the Reddit/Openwrt community. In fact I only found it by complete accident doing a google search. Hit the jackpot in this case tho.
I'll send details of performance etc when I get home (which is very, very good BTW!)
I totally get the usage cases dnsmasq vs adblock & simple. Ie wanting to manage client DNS. However I'm pretty liberal with that, if guests want to use their own DNS, go for it.
The below graph is anecdotal. But context - I'm now running the full OISD blocklist, Layer Cake SQM, ~15 devices, 3x IP motion cameras, IOT & Guest WLAN, 100/20 (Australia speeds) on R7800. For comparison the full OISD blocklist on Adblock with 21.02 would hover around 20% on one CPU and could take literal seconds to load a webpage. The full OISD blocklist on dnsmasq is also noticeably faster than Adblock on 21.02 with OISD basic (webpage load speeds). No other graphs sorry, just what I described here.
I got home around 17:00, web browsing / searching etc. Video chat started around 17:35. All in all this router/config is just idling along. This is easily the fastest setup I've run....pages just load quickly.
One thing to note and IDKW, but after running the dnsmasq command on bootup, WLAN1-1 and WLAN1-2 stop providing DHCP. I just run "/etc/init.d/network restart" straight after in local startup, and all is good.
edit: forgot to mention the graph looks almost identical without dnsmasq/oisd_full. Close to zero impact
Also saw it over on Reddit, thanks for sharing here.
Just sharing what I've configured in Startup and Scheduled Tasks:
mkdir -p /tmp/dnsmasq.d && wget https://dnsmasq.oisd.nl -O /tmp/oisd.txt && mv /tmp/oisd.txt /tmp/dnsmasq.d/oisd.txt && /etc/init.d/dnsmasq restart
mkdir -p /tmp/dnsmasq.d ensures the folder exists
wget https://dnsmasq.oisd.nl -O /tmp/oisd.txt downloads the list to /tmp/oisd.txt (a temporary file)
mv /tmp/oisd.txt /tmp/dnsmasq.d/oisd.txt moves the temporary file to the actual location. This is so that if the wget download fails it doesn't load an empty file. If the download failed, this command will fail as there won't be any /tmp/oisd.txt file to move.
/etc/init.d/dnsmasq restart restarts the dnsmasq service with the updated list
And for those who aren't familiar, anything in the /tmp directory is stored on RAM (not non-volatile storage), which is good for the longevity of the device.
I've paired the above with this firewall rule to force all devices' plain 53 connections to be filtered: https://openwrt.org/docs/guide-user/firewall/fw3_configurations/intercept_dns
Ah, you meant zero impact in CPU load, I thought you meant zero impact on speed of resolution requests (which was an issue with the older dnsmasq versions for large servers lists).
@jackiechun does some minimum amount of control need to be put into place to ensure the source of the information isn't malicious and poisoning DNS entries?
Maybe I'm a nervous Nelly, because I never heard of oisd.nl before?
Example1: Maybe grep -v out any entries that map FQDN to an IP??
It's a popular blocklist, integrated in AdGuard DNS, NextDNS, and ControlD: https://oisd.nl/howto
But you can use the above method with your own or any other blocklist if you want.
It looks like the OISD list almost tripled in size overnight and my OpenWRT was not able to handle it, did you encounter this as well?
You are right, the full dnsmasq file went from ~10mb to ~25mb. Hopefully this is a temporary issue?
Using an extra 55mb ram on r7800, but still have 280mb free and seems to be running fine.
Well looks like OISD full dnsmasq file is back at it's usual ~10mb on the website....
Thanks for the update! This experience made me realize that my modest Archer C7 is too old to handle these possible fluctuations so I've decided to generate my own static dnsmasq blocklist in place of the OISD list, and changed to an adblocking public dns.
I wrote earlier that with the startup script I lost DHCP for WLAN1-1 and WLAN1-2 (my guest and IOT WLAN's) and was using "/etc/init.d/network restart" at startup as a workaround. Now just sharing that I'm using sleep 60 before the script (last in local startup) and everything is working 100% without a network restart at startup. IMO sleep 60 is the nicer solution for me...
@jackiechun Thanks for that script with sense checking - good idea. I guess you could add a file size check and ignore if it's oversized - just an idea tho. I have used both NextDNS and Adguard before in round robin to accommodate the 300k dns lookup per month limit both of them have.
I've decided to give it another try: the allure of using Cloudflare/Quad9 DNS servers was too great.
My solution: simple bash script that will pre-process the blocklist and push to Github if < 400,000 entries (arbitrarily chosen), and I've modified my command to download that blocklist from Github. Currently the list is ~340,000, when it ballooned 2 weeks ago it was ~ 950,000.
A bonus to this is I can append domains I want blocked (such as autoupdates for my Amazon Firesticks). Will run and see how it goes, I'll only report back if there are any issues.