So basically just remove every line that is not in this format below(?):
address=/[a-z.]/#
And I like your idea about checking a few sites.
I am also keen to setup that as a fallback position we go back to the last working modified oisd.txt rather than falling back to nothing.
Also I like having 'get-oisd.sh' because it means a user can manually fetch the latest and greatest from the command line by calling the script.
Domain names can also include numbers and hyphens, not to mention IDN names will include non-ASCII characters.
Ah yes of course thanks. My mind is exhausted after working out how to kill another patent.
I like the idea of the .sh script, should be easier to read/understand/edit rather than all in one line as is currently.
Falling back to the last working list is a good idea. Even if that doesn't happen it should only be 24 hours until the next cron update anyway.
Just adding address=/malware.com/ is legitimate too and returns NXDOMAIN. The # returns null 0.0.0.0
Malformed list might prevent dnsmasq from starting. Even non-malformed list may contain let's say com or net which will prevent you from resolving any .com or .net domain.
simple-adblock is essentially the shell script you're looking for. 
I thought you said above it'd choke on this oisd list?
Hmmm I wonder how well oisd full and adblock or simple would run on 22.03? I tried adblock / oisd full on 21.02 and it lagged bad for any/all dns lookups. Plus router cpu was @20% all the time.
Edit: I'll try them out on the weekend just to see....
Yeah makes sense. Don't want to reinvent the wheel and perhaps it's just simple-adblock needs updating to reflect changes to dnsmasq? So that it benefits from the new faster handling we have been able to leverage in this thread? @stangri?
It was the DNSMasq v2.86 update in OpenWrt v22.03 that made the improvement: https://thekelleys.org.uk/dnsmasq/CHANGELOG
I said it might, depending on the router. The more processing you want to run on it, the higher are RAM requirements (because OpenWrt's temporary folder is in ramdisk).
Please elaborate on updates needed to be done to simple-adblock to reflect changes to dnsmasq.
Once simple-adblock has finished running, it's all dnsmasq which is doing resolution. Dnsmasq doesn't (or didn't used to) handle large lists of servers from external file, that's why simple-adblock supports producing different outputs for dnsmasq config, you can check README to find out more.
But if anything, dnsmasq + list produced by simple-adblock would be more efficient than dnsmasq + list off internet, unless the list off the internet been optimized to remove some unnecessary 3rd level domains, as the simple-adblock's list would be smaller.
To make it easier:
simple-adblock README
simple-adblock package
user guide
forum thread
@stangri does the script allow daily list refreshes or only on boot? All of my routers are in production and I don't have a spare at the moment to test.
You'll need to add a cron job for /etc/init.d/simple-adblock dl to redownload/re-process everything.
PS. I've started replying because the subject of writing a new shell script came up, that's why I suggested simple-adblock as a template already pre-set to work as init script with some filters already defined.
Ah, I got confused by:
It seems I wrongly assumed that the approach listed in this thread would leverage new functionality in dnsmasq that is not leveraged in simple-adblock, and that therefore for the same sized-list simple-adblock would be slower.
So for same list simple-adblock and this solution would give same speed, right?
So assuming I was wrong then the case for putting together a bash script based on what is discussed in this thread is rather undermined, and I do take your point here:
Looking at the readme looks like you've done some really nice work here. I like just simple init.d script.
Something I can't see from the readme is what are the default lists that simple-adblock utilises?
Oh, good point, maybe I should put a link to the config file in the github repo!
This is the default config, I hope things are self-explanatory, if you have further suggestions for improvements, let me know:
https://github.com/openwrt/packages/blob/master/net/simple-adblock/files/simple-adblock.conf
Hey stangri,
For simple-adblock, would processing/filtering the oisd dnsmasq file be faster than say hosts file to dnsmasq format? Ie Would there be less formatting conversion etc? If so, whats the chances of getting a dnsmasq file option added into simple-adblock?
Oh, good question, I haven't tested, but since both domains and hosts files are processed with the simple sed filters, if there is a difference it would be negligible. The filters themselves are found in those two lines: https://github.com/openwrt/packages/blob/master/net/simple-adblock/files/simple-adblock.init#L63-L64
The look big but compared to more complex parsing which can be done by awk, they run fast. If you want, you can download various domains and hosts block files and run those filters on them with the time command to see how long they take on your router. Let me know if you want any pointers on that, as I myself am curious of the results.
What's taking longest currently is allow-listing, the longer/bigger your allow list is, the longer it takes.
simple-adblock already supports all of the dnsmasq options of blocking things. I believe what you call "dnsmasq file option" is the dnsmasq.servers which is default.
Cool thanks, I'll have a play around on the weekend and see how it goes on a r7800. What I'm referring to though, is using the oisd 'dnsmasq' file as an input file into simple-adblock. oisd has a specific file for dnsmasq. At least, I didn't see that as an option so far, only hosts and domain files from my brief look over
Ah, right, I see. So in order to optimize it, it would still have to be converted to the domains list, so if the same source has just the domains list it would be faster.
In my mind, the only way this option would be useful if you don't have any other lists enabled and don't want any processing done, just download and check for it not being malformed. Is it also how you see it used? If not, please elaborate.
Yes spot on, this is exactly what I mean. Oisd is designed to be quote 'might the only Blocklist you need'. So I'm using it exclusively, as it blocks as much as possible without breaking websites. I believe oisd says 'grandparents and girlfriends' approved due to this. Lol