I thought you said above it'd choke on this oisd list?
Hmmm I wonder how well oisd full and adblock or simple would run on 22.03? I tried adblock / oisd full on 21.02 and it lagged bad for any/all dns lookups. Plus router cpu was @20% all the time.
Edit: I'll try them out on the weekend just to see....
1 Like
Yeah makes sense. Don't want to reinvent the wheel and perhaps it's just simple-adblock needs updating to reflect changes to dnsmasq? So that it benefits from the new faster handling we have been able to leverage in this thread? @stangri?
1 Like
It was the DNSMasq v2.86 update in OpenWrt v22.03 that made the improvement: https://thekelleys.org.uk/dnsmasq/CHANGELOG
I said it might, depending on the router. The more processing you want to run on it, the higher are RAM requirements (because OpenWrt's temporary folder is in ramdisk).
Please elaborate on updates needed to be done to simple-adblock to reflect changes to dnsmasq.
Once simple-adblock has finished running, it's all dnsmasq which is doing resolution. Dnsmasq doesn't (or didn't used to) handle large lists of servers from external file, that's why simple-adblock supports producing different outputs for dnsmasq config, you can check README to find out more.
But if anything, dnsmasq + list produced by simple-adblock would be more efficient than dnsmasq + list off internet, unless the list off the internet been optimized to remove some unnecessary 3rd level domains, as the simple-adblock's list would be smaller.
2 Likes
To make it easier:
simple-adblock README
simple-adblock package
user guide
forum thread
@stangri does the script allow daily list refreshes or only on boot? All of my routers are in production and I don't have a spare at the moment to test.
You'll need to add a cron job for /etc/init.d/simple-adblock dl to redownload/re-process everything.
PS. I've started replying because the subject of writing a new shell script came up, that's why I suggested simple-adblock as a template already pre-set to work as init script with some filters already defined.
1 Like
Ah, I got confused by:
It seems I wrongly assumed that the approach listed in this thread would leverage new functionality in dnsmasq that is not leveraged in simple-adblock, and that therefore for the same sized-list simple-adblock would be slower.
So for same list simple-adblock and this solution would give same speed, right?
So assuming I was wrong then the case for putting together a bash script based on what is discussed in this thread is rather undermined, and I do take your point here:
Looking at the readme looks like you've done some really nice work here. I like just simple init.d script.
Something I can't see from the readme is what are the default lists that simple-adblock utilises?
1 Like
Oh, good point, maybe I should put a link to the config file in the github repo!
This is the default config, I hope things are self-explanatory, if you have further suggestions for improvements, let me know:
https://github.com/openwrt/packages/blob/master/net/simple-adblock/files/simple-adblock.conf
1 Like
Hey stangri,
For simple-adblock, would processing/filtering the oisd dnsmasq file be faster than say hosts file to dnsmasq format? Ie Would there be less formatting conversion etc? If so, whats the chances of getting a dnsmasq file option added into simple-adblock?
1 Like
Oh, good question, I haven't tested, but since both domains and hosts files are processed with the simple sed filters, if there is a difference it would be negligible. The filters themselves are found in those two lines: https://github.com/openwrt/packages/blob/master/net/simple-adblock/files/simple-adblock.init#L63-L64
The look big but compared to more complex parsing which can be done by awk, they run fast. If you want, you can download various domains and hosts block files and run those filters on them with the time command to see how long they take on your router. Let me know if you want any pointers on that, as I myself am curious of the results.
What's taking longest currently is allow-listing, the longer/bigger your allow list is, the longer it takes.
simple-adblock already supports all of the dnsmasq options of blocking things. I believe what you call "dnsmasq file option" is the dnsmasq.servers which is default.
Cool thanks, I'll have a play around on the weekend and see how it goes on a r7800. What I'm referring to though, is using the oisd 'dnsmasq' file as an input file into simple-adblock. oisd has a specific file for dnsmasq. At least, I didn't see that as an option so far, only hosts and domain files from my brief look over
Ah, right, I see. So in order to optimize it, it would still have to be converted to the domains list, so if the same source has just the domains list it would be faster.
In my mind, the only way this option would be useful if you don't have any other lists enabled and don't want any processing done, just download and check for it not being malformed. Is it also how you see it used? If not, please elaborate.
Yes spot on, this is exactly what I mean. Oisd is designed to be quote 'might the only Blocklist you need'. So I'm using it exclusively, as it blocks as much as possible without breaking websites. I believe oisd says 'grandparents and girlfriends' approved due to this. Lol
1 Like
may need the addition of the nsfw list.
1 Like
Great, that shouldn't be very hard to implement, I'm sure I can push the update to the repo early next week. Are you willing to test the builds before then?
2 Likes
Liking the luci interface! @stangri out of the lists here:
which would you recommend enabling if not just defaults as you have it? The OISD list is right at the bottom. Does that mean if I enable it I don't need to enable the others? Overlapping entries are removed automatically right?
Also I have selective DNS hijacking. I can presumably disable force all lan clients. But maybe it'd be nice to allow overrides for certain lan clients to specific DNS servers? Or perhaps it just gets too complicated to generalize.
1 Like
This is my config (with personal allow/block-lists removed) from the x86_64-based router, but I believe I used the same config on the EdgeRouter-X as well:
config simple-adblock 'config'
option dns 'dnsmasq.servers'
option dns_instance '0'
option force_dns '1'
list force_dns_port '53'
list force_dns_port '853'
option canary_domains_icloud '0'
option canary_domains_mozilla '0'
option download_timeout '10'
option curl_retry '3'
option parallel_downloads '1'
option debug '0'
list allowed_domain 'cdn.jsdelivr.net'
list allowed_domain 'melmac.net'
list blocked_hosts_url 'https://adaway.org/hosts.txt'
list blocked_hosts_url 'https://cdn.jsdelivr.net/gh/hoshsadiq/adblock-nocoin-list/hosts.txt'
list blocked_hosts_url 'https://pgl.yoyo.org/as/serverlist.php?hostformat=hosts&showintro=1&mimetype=plaintext'
list blocked_hosts_url 'https://winhelp2002.mvps.org/hosts.txt'
list blocked_hosts_url 'https://someonewhocares.org/hosts/hosts'
list blocked_hosts_url 'http://sysctl.org/cameleon/hosts'
list blocked_hosts_url 'https://cdn.jsdelivr.net/gh/StevenBlack/hosts/hosts'
list blocked_hosts_url 'https://hosts.oisd.nl/'
option enabled '1'
option boot_delay '30'
option compressed_cache '1'
option config_update_enabled '1'
option config_update_url 'https://source.openwrt.melmac.net/simple-adblock/files/simple-adblock.conf.update'
option verbosity '1'
list blocked_domains_url 'https://cdn.jsdelivr.net/gh/AdguardTeam/cname-trackers@master/combined_disguised_trackers_justdomains.txt'
The config-update-url in my config file is set to pull the updates from my private repo, you may want to leave it with the github openwrt url instead.
1 Like
yes, disable force_dns in simple-adblock if you want to use your own dns hijacking rules.
LOL. That really depends what your grandparents and girlfriends are into 