@account4538 do you think this would catch dangerous rogue entries?
It certainly catches:
address=/apple.com/10.0.0.1
address=/appleiphonecell.com/10.0.0.1
address=/airport.us/10.0.0.1
address=/akamaiedge.net/10.0.0.1
i.e. any redirect to an IP rather than the null address #.
Any other case we need to address (no pun intended!)?
It pulls out examples like this as well:
address=/test.com/a
address=/test.com/1
address=/test.com/0.0.0.0
address=/test.com/?
Even though none are necessarily dangerous, you could argue that if the oisd.txt list included these, something may have gone wrong with that particular list compile. I've searched through several list versions, and happily have not found anything at all out of order.
What I do want to include is something to rip this type of entry out (note the 3x "/"). I'll work on it.
address=/oisdmytest.com/1.1.1.1/#
Edit: this returns 0.0.0.0 anyway, so not really an issue. But wouldn't hurt to test regardless.
Name: oisdmytest.com
Addresses: ::
0.0.0.0
Short answer? 0.0.0.0 is better. its an answer rather than a don't know which can lead to more queries to find out or trying to use an alternate DNS server.
NXDOMAIN may cause your device to fall back to some other DNS resolver, whereas 0.0.0.0 is an answer so your device will stop asking.
Also android tends to go looking again if you use NXDOMAIN. See this GitHub thread about it.
Regarding trust of blocking/filter lists. All you can do is try the lists and see if they work for your purposes. Some lists are very well run with dedicated reviewers and a process to get wrongly blocked addresses removed/reviewed.
I've included a few I use below from my AGH thread. They can be used with dnsmasq, AdGuard Home, Pihole or simple Adblock.
Filter Sites:
https://firebog.net/
https://oisd.nl/ For AdGuardHome you must use abp.oisd.nl
This has a major collection of blocklists allowing you to pick and choose and even gives them ratings for safe usage.
This set of lists are for keeping your smart tv/devices from spying and reporting back.
This is a very important list. It is a whitelist to always allow these sites. Its more useful for programs like AdGuard Home and PiHole as these allow you to put in exclusions like this but its still worth knowing about.
Last thing? Using AdGuard Home or PiHole while initially bit more complex to setup, does give you greater handling of blocklists and exclusions. It makes customising your needs simpler and also gives you a dashboard to check on.
Dnsmasq with a list its defiantly useful for smaller routers with low ram/disk space as an intermediary step but I still recommend AGH or PiHole, especially if you are using for parental purposes.
Forgot 2 important ones I use. First one blocks Phising sites. second is active digital threats.
The good news is that oisd full already combines these sites you listed, or a variant of them, plus many more.
And then it applies a whitelist over the top also:
"Where most other blocklists go for the "block ads/trackers and when something breaks it's up to the user to figure out what to whitelist"-approach .. This list prioritizes functionality over blocking."
Indeed. but I prefer to be sure and treat these lists like Venn diagrams. As long as there is overlap its good (and any dupes will be stripped out when they are built into a master list by AGH.) Its actually why I don't use OISD and just use source lists instead.
My reasoning for this is AGH will tell you which blocklist has blocked something when you check the logs. Thus its easier to find the offending list and track it down vs one big master list. Just a personal choice. That being said its been a while since I've had issues. I believe last set of exceptions I had to include was for streaming IPTV (just needed a domain enabling). Also its because originally I used a 128mb router and using huge blocklists would cause out of memory errors. I could get about 140k of lists before it would die.
I've just listed all the ones I use and reference sites as a new post on my AGH thread if you are curious.
Yeah cool I do like to understand what options are out there. And there is definitely some upsides to running dedicated software ie logging as you mentioned.
Also benefits to running just a list, which is no additional software needed.
Its one reason I try to lay out my reasonings and references. Help people understand the why and how etc.
As I said earlier I originally started out with 128mb router so you have to be so careful to make sure you can do blocking without killing performance or memory.
Approaching from an engineering view I've always tried to keep it small but functional. There are some on pihole sites who have 4million plus sites in blocklists. Now that is ok if your router is powerful and has memory but it can be rather overkill. For instance if you aren't in china why include their blocklists?
I originally started with simple blocklist and curated lists, I later moved to a pihole and finally to AGH. But I now have a router with 4gb of ram so don't have to be as careful. Others however will need other options and understanding how to fit a solution for smaller routers is important.
Also dnsmasq and lists for a small light travel router is ideal and improves privacy and protection while traveling. Hostile wifi points are a always there threat.
And to add, there can be more reasons choosing not to run certain software over just router capabilities 
Some final, I hope!, edits to my one liner for anyone that wishes to use this. Lots of other options in this thread to choose from also of course. Now a rolling log kept at ~999 lines, stored in /tmp (ram). Log is just a crude screen dump but contains enough to troubleshoot if ever needed. Record the number of oisd lines before and after sed filter. Variables to shorten everything up a bit.
I'm personally still running ping connectivity check after applying oisd and restarting dnsmasq. At least until someone can definitively say it's impossible to have dnsmasq running, yet no internet connectivity due to oisd list.
Using https-over-dns and/or plenty-o-ram
OIL=/tmp/oisd.log; echo "$(tail -999 $OIL)" > $OIL; { date; OIF=/tmp/dnsmasq.d/oisd.txt; curl --max-filesize 20971520 --max-time 60 --retry 3 --url https://dnsmasq.oisd.nl/ --output $OIF && head -n 20 $OIF && wc -l $OIF && sed -i -E '\~^address=/[[:alnum:]][[:alnum:].-]+/[\d35]$~!d' $OIF && wc -l $OIF && /etc/init.d/dnsmasq restart; sleep 60; { pgrep -x dnsmasq && { ping -c 1 cloudflare.com || ping -c 1 quad9.net ;} ;} || { rm $OIF; /etc/init.d/dnsmasq restart ;} ;} 2>&1 | tee -a $OIL
No https-over-dns or low ram:
OIL=/tmp/oisd.log; echo "$(tail -999 $OIL)" > $OIL; { date; OIF=/tmp/dnsmasq.d/oisd.txt; curl --max-filesize 20971520 --max-time 60 --retry 3 --url https://dnsmasq.oisd.nl/ --output $OIF && head -n 20 $OIF && wc -l $OIF && sed -i -E '\~^address=/[[:alnum:]][[:alnum:].-]+/[\d35]$~!d' $OIF && wc -l $OIF && /etc/init.d/dnsmasq restart; sleep 60; rm $OIF; { pgrep -x dnsmasq && { ping -c 1 cloudflare.com || ping -c 1 quad9.net ;} ;} || /etc/init.d/dnsmasq restart ;} 2>&1 | tee -a $OIL
if you want to remove ping check from either option, delete this section:
&& { ping -c 1 cloudflare.com || ping -c 1 quad9.net ;}
@Lynx
I got some help from fine peeps at stackoverflow.com
1st pass check anything starting with 'address=' and not ending with '/#'
'\|^address=|!d;\|/#$|d'
I'm so sorry somehow didn't originally paste the $ for end of line
Second pass filter will behave exactly as I want now
-E '\~^address=/[[:alnum:]][[:alnum:].-]+/#$~!d'
-E = extended options
and delete lines such as:
address=/.com/#
address=test.com/#
address=/test/com/#
address=/test.com/1.1.1.1/#
address=/test.com/1.1.1.1
address=/test.com/ (this line isn't actually a problem, but oisd dnsmasq uses /# anyway)
OK @Wizballs how does this look to you now:
If a rogue element is identified in the first pass then processing is discontinued and first rogue element is printed out. Otherwise the file is processed according to the second pass.
Hey, so '\|^address=|!d;\|/#$|d;q' is sent to $rogue_element, if $rogue_element =/= zero then there is an issue. Looks quite tidy really.
I do want to tidy up the first check more eg what do you think about this? A lot like the second pass filter but opposite. It would pick up more than currently.
if start with address=(/ must follow)(alnum's must follow)(alnum's or .- must follow)(must end with /#)
Then second pass would just clean out random bits eg if there just happened to be a random text line.
Having said that, haven't seen a bad line in oisd yet. But checks are a must for running!
But I do appologise I might be a bit quiet for the next ~10 days. RL work (finance) is about to get pretty crazy unfortunately with a few things due.
I just added extra dnsmasq checks:
Finance? Crazy times at the moment right? My VWRP doesn't look so great anymore.
You can use && to combine commands. It will stop on the first error.
So if grep fails it wont ping and so on:
{ pgrep -x dnsmasq && ping -c 1 google.com && ping -c 1 cloudflare.com ;} &>/dev/null
edit: If needed you can add a return $?
From my testing this seems to be working pretty well now:
Very easy to install - just put service file into /etc/init.d/ and enable and start service.
And optionally put the following entry into 'Scheduled Tasks':
0 5 * * * adblock-oisd enabled; [[ $? -eq 0 ]] && /etc/init.d/adblock-oisd start
to update at 5am every morning (in dependence upon the various checks).
Maybe also put
/etc/unit.d/adblock-oisd start
in /etc/rc.local so everytime the router boots the oisd list gets downloaded.
Edit also on your github, the first line of the installation instructions is missing /main/ between your username and the repo.
Actually one isn't better than the other, it depends.
I have seen some cases where since 0.0.0.0 is an answer it continually tries to keep connecting to 0.0.0.0 like a server is down or unresponsive. Apple recommends using NXDOMAIN for their devices.
In other cases, NXDOMAIN will cause a retry or fall back like you mentioned with some Android devices.
It depends on the software & application. You can find answers for both ways.
Hey Lynx nice work! Looking forward to giving it a spin as soon as I get the time. Will report back....
Cool. I have it running all the time now and it updates every 5am just fine. I like that this service script solution: 'adblock-oisd' - inspired by this thread - has a tiny footprint at less than 3KB and I believe more sanity checks and safeguards than the heavier alternatives. And of course it can be improved further with time.