Adblock-oisd : 22.03 allows you to use huge blocklists with dnsmasq

Those are blocked by just the oisd hosts file for me.

1 Like

That's the wrong file, the correct file for dnsmasq config should be /tmp/dnsmasq.d/simple-adblock, please update to the most recent simple-adblock.

1 Like

Ok my bad sorry not sure how I overlooked the dns service option

EDIT: I have edited this post, as everything is working well now with https://dnsmasq.oisd.nl/. Thankyou. Seems the issue stopping everything working may have been changing Download time-out (in seconds) from the default of 10, to 60.

1 Like

Oh, no, I overlooked that for the external dnsmasq config the dns type needs to be changed and I believe the first build I pushed had that bug. From then on, if the use of the external config file is detected, the dns setting is overridden to dnsmasq.conf on start/in WebUI reporting.

2 Likes

My updated script:

{ date && curl --max-filesize 20971520 --max-time 60 --retry 3 --url https://dnsmasq.oisd.nl/ --output /tmp/dnsmasq.d/oisd.txt && head -n 20 /tmp/dnsmasq.d/oisd.txt && sed -i '\|^address=/[[:alnum:]]|!d;\|/[\d35]$|!d' /tmp/dnsmasq.d/oisd.txt && /etc/init.d/dnsmasq restart; sleep 60; rm /tmp/dnsmasq.d/oisd.txt; { pgrep -x dnsmasq && { ping -c 1 1.1.1.1 || ping -c 1 9.9.9.9 ;} ;} || /etc/init.d/dnsmasq restart ;} 2>&1 | tee /tmp/oisd.log

  • Writes all output to /tmp/oisd.log
  • sed command to clean oisd.txt file (sed already part of openwrt, no additional package needed)
  • Removes /tmp/dnsmasq.d/oisd.txt after dnsmasq is restarted. As this file is in ram, gives back more free ram
  • Must pass dnsmsaq is running, and at least 1 of the 2 ping tests must pass, otherwise dnsqmasq is restarted without /tmp/dnsmasq.d/oisd.txt. Using both of my dns providers for the ping test (1.1.1.1 & 9.9.9.9)

This is just about reaching the character limit for one line (at least, character limit via putty anyway). Anything fancier would need to go into a ".sh" file.

EDIT: Fixed up the sed command so will run in both startup and crontab. # was the culprit stopping cron from running, so simply replaced with character code \d35

We really need a dnsmasq blocklist UI in LUCI. Something simple just for blocklists. The one in Fresh Tomato is great.

2 Likes

Nice, with so much in the command maybe a script would be the better option here.

Just a note that if DNSMasq doesn't start you can still ping.

I mean, there is simple-adblock. There is a best option for everybody. My script just has a few things for my use case

  • max file size
  • deletes /tmp/dnsmasq/oisd.txt after dnsmasq restart, to free up ram
  • tweaked sed command

hahah yep, that is pretty much the limit of this one-liner! To do more fancy stuff like fallback to last good list, then needs a .sh script. Only a couple of extra steps to get that running like creating the script and enabling execute permission.

This is the best one-liner I can do for a simple copy/paste :wink:

This updated script checks two things:

  1. dnsmasq is running
  2. at least one of the dns servers can be pinged

if either 1 or 2 fail, then delete oisd.txt and restart dnsmasq

PS just updated the script above to correct the sed command

My 2 cents:

wget --timeout 60 https://dnsmasq.oisd.nl -O- | sed -n '\,^address=.*#,s,#.*$,,p' > /tmp/dnsmasq.d/oisd.txt

EDIT: removed the newline in "address"

That sed doesn't seem to do anything.

1 Like

Ahh.. yes there was a newline in there :wink: corrected

root@OpenWRT:~# wget --timeout 60 https://dnsmasq.oisd.nl -O- | sed -n '\,^address
=.*#,s,#.*$,,p' > /tmp/dnsmasq.d/oisd.txt
Downloading 'https://dnsmasq.oisd.nl'
Connecting to 51.178.86.254:443
Writing to stdout
-                    100% |*******************************|  8186k  0:00:00 ETA
Download completed (8382876 bytes)
root@OpenWRT:~# head -n 10 /tmp/dnsmasq.d/oisd.txt 
address=/0--foodwarez.da.ru/
address=/0-0-0-0-0-0-0-0-0-0-0-0-0-18-0-0-0-0-0-0-0-0-0-0-0-0-0.info/
address=/0-0-0-0-0-0-0-0-0-0-0-0-0-33-0-0-0-0-0-0-0-0-0-0-0-0-0.info/
address=/0-0-0-0-0-0-0-0-0-0-0-0-0-37-0-0-0-0-0-0-0-0-0-0-0-0-0.info/
address=/0-000.store/
address=/0-24bpautomentes.hu/
address=/0-29.com/
address=/0-36.binance.com/
address=/0-74.us.api.binance.com/
address=/0-800-email.com/
root@OpenWRT:~# 
1 Like

Here is my initial draft:

https://github.com/lynxthecat/adblock-oisd/blob/main/adblock-oisd

Written as a service and 'service adblock-oisd start' will download and setup dnsmasq with a new oisd.txt file. Checks are performed and the script will either fallback to a previous oisd.txt file if available or no oisd.txt in dependence upon the outcome of the checks.

'service adblock-oisd start' could be added in Scheduled Tasks, but ideally we would first check if the service is enabled. How can we check for that?

It will likely need some tweaking, but hopefully this is a good start.

2 Likes

Hey bigsmile, does this run for you in crontab? I couldn't get mine to run in crontab if it had # symbol anywhere in the line, which is why I substituted # for character code \d35 in sed command.

also this is allowing malformed items through:

address=.com/#    >>    address=.com/
address=/.com/#   >>    address=/.com/

live sed editor here: https://sed.js.org/

Nice one! When I get time I might give that a spin.
I know it's a draft, but that sed command is allowing things through:
address=/_.com/#
address=/.com/#
address=/test/#.com

'\|^address=/[[:alnum:]]|!d;\|/[\d35]$|!d'
or
'\|^address=/[[:alnum:]]|!d;\|/#$|!d'

will block those examples
tested here https://sed.js.org/

The \d35 did not work for me (at least not on the command prompt), go figure :smiley: So I have to do some additional testing with the crontab. But to be honest I was hoping someone else would figure it out :wink:

This gives:

sed: -e expression #1, char 1: unknown command: `|'
1 Like

Arr sorry must have copied the command over wrong somehow. Should be:

'\|^address=/[[:alnum:]]|!d;\|/[\d35]$|!d'
or
'\|^address=/[[:alnum:]]|!d;\|/#$|!d'
1 Like

@Wizballs I'm not familiar with sed syntax - could you explain how:

'\|^address=/[[:alnum:]]|!d;\|/#$|!d'

works?

1 Like

I'm still learing also - sed is complex but so versatile.
Syntax and formatting aside:
Delete anything not starting with address=/(followed by one alphanumberic character)
Delete anything not ending with /# (if there are lines with whitespaces after /# they will be deleted also with this command as is. Not sure if this should be accounted for, but oisd.txt doesn't seem to have whitespaces after anyway)

So it's pretty strict but better than having dnsqmasq not running

1 Like