Simple-adblock: fast, lean and fully uci/luci configurable AdBlocking


#123

I have some more information. In attempting to block embedded YouTube video ads, I added the raw domain list here:

Seems like the αΊ‘ unicode character from one of the domains is what broke it:

Thu Aug 23 19:10:43 2018 user.notice simple-adblock [6023]: Reloading simple-adblock 1.6.3-5...
Thu Aug 23 19:10:43 2018 daemon.info dnsmasq[5892]: exiting on receipt of SIGTERM
Thu Aug 23 19:10:43 2018 user.notice dnsmasq: DNS rebinding protection is active, will discard upstream RFC1918 responses!
Thu Aug 23 19:10:43 2018 user.notice dnsmasq: Allowing 127.0.0.0/8 responses
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: started, version 2.80test3 cachesize 150
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: DNS service limited to local subnets
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: compile time options: IPv6 GNU-getopt no-DBus no-i18n no-IDN DHCP no-DHCPv6 no-Lua TFTP no-conntrack no-ipset no-auth no-DNSSEC no-ID loop-detect inotify dumpfile
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq-dhcp[6149]: DHCP, IP range 192.168.1.100 -- 192.168.1.249, lease time 12h
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using local addresses only for domain test
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using local addresses only for domain onion
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using local addresses only for domain localhost
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using local addresses only for domain local
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using local addresses only for domain invalid
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using local addresses only for domain bind
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 2606:4700:4700::1001#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 2606:4700:4700::1111#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 1.0.0.1#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 1.1.1.1#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using local addresses only for domain lan
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: reading /tmp/resolv.conf.auto
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using local addresses only for domain test
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using local addresses only for domain onion
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using local addresses only for domain localhost
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using local addresses only for domain local
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using local addresses only for domain invalid
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using local addresses only for domain bind
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 2606:4700:4700::1001#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 2606:4700:4700::1111#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 1.0.0.1#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 1.1.1.1#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using local addresses only for domain lan
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 1.1.1.1#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 1.0.0.1#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 2606:4700:4700::1111#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 2606:4700:4700::1001#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 1.1.1.1#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 1.0.0.1#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 8.8.8.8#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 8.8.4.4#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 2606:4700:4700::1111#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 2606:4700:4700::1001#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 2001:4860:4860::8888#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 2001:4860:4860::8844#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: read /etc/hosts - 4 addresses
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: read /tmp/hosts/odhcpd - 0 addresses
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: read /tmp/hosts/dhcp.cfg01411c - 10 addresses
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq-dhcp[6149]: read /etc/ethers - 0 addresses
Thu Aug 23 19:10:46 2018 daemon.err dnsmasq[6149]: cannot read /tmp/adb_list.overall: No such file or directory
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using local addresses only for domain test
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using local addresses only for domain onion
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using local addresses only for domain localhost
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using local addresses only for domain local
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using local addresses only for domain invalid
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using local addresses only for domain bind
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 2606:4700:4700::1001#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 2606:4700:4700::1111#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 1.0.0.1#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 1.1.1.1#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using local addresses only for domain lan
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 1.1.1.1#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 1.0.0.1#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 2606:4700:4700::1111#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 2606:4700:4700::1001#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 1.1.1.1#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 1.0.0.1#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 8.8.8.8#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 8.8.4.4#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 2606:4700:4700::1111#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 2606:4700:4700::1001#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 2001:4860:4860::8888#53
Thu Aug 23 19:10:46 2018 daemon.info dnsmasq[6149]: using nameserver 2001:4860:4860::8844#53
Thu Aug 23 19:10:46 2018 user.notice simple-adblock [6023]: Low free memory, restarting dnsmasq...[βœ“]
Thu Aug 23 19:10:46 2018 user.notice simple-adblock [6023]: [DL] Blocked Hosts: zeustracker.abuse.ch [βœ“]
Thu Aug 23 19:10:46 2018 user.notice simple-adblock [6023]: [DL] Blocked Hosts: adaway.org [βœ“]
Thu Aug 23 19:10:46 2018 user.notice simple-adblock [6023]: [DL] Blocked Domains: ransomwaretracker.abuse.ch [βœ“]
Thu Aug 23 19:10:46 2018 user.notice simple-adblock [6023]: [DL] Blocked Hosts: raw.githubusercontent.com [βœ“]
Thu Aug 23 19:10:46 2018 user.notice simple-adblock [6023]: [DL] Blocked Hosts: www.malwaredomainlist.com [βœ“]
Thu Aug 23 19:10:47 2018 user.notice simple-adblock [6023]: [DL] Blocked Domains: s3.amazonaws.com [βœ“]
Thu Aug 23 19:10:47 2018 user.notice simple-adblock [6023]: [DL] Blocked Domains: raw.githubusercontent.com [βœ“]
Thu Aug 23 19:10:47 2018 user.notice simple-adblock [6023]: [DL] Blocked Domains: ssl.bblck.me [βœ“]
Thu Aug 23 19:10:47 2018 user.notice simple-adblock [6023]: [DL] Blocked Domains: s3.amazonaws.com [βœ“]
Thu Aug 23 19:10:47 2018 user.notice simple-adblock [6023]: [DL] Blocked Domains: s3.amazonaws.com [βœ“]
Thu Aug 23 19:10:47 2018 user.notice simple-adblock [6023]: [DL] Blocked Hosts: pgl.yoyo.org [βœ“]
Thu Aug 23 19:10:47 2018 user.notice simple-adblock [6023]: [DL] Blocked Domains: mirror1.malwaredomains.com [βœ“]
Thu Aug 23 19:10:48 2018 user.notice simple-adblock [6023]: [DL] Blocked Hosts: www.mvps.org [βœ“]
Thu Aug 23 19:10:49 2018 user.notice simple-adblock [6023]: [DL] Blocked Hosts: someonewhocares.org [βœ“]
Thu Aug 23 19:10:50 2018 user.notice simple-adblock [6023]: [DL] Blocked Domains: dshield.org [βœ“]
Thu Aug 23 19:10:50 2018 user.notice simple-adblock [6023]: Sorting combined list [βœ“]
Thu Aug 23 19:10:57 2018 user.notice simple-adblock [6023]: Optimizing combined list [βœ“]
Thu Aug 23 19:10:57 2018 user.notice simple-adblock [6023]: Whitelisting domains [βœ“]
Thu Aug 23 19:10:57 2018 user.notice simple-adblock [6023]: Formatting merged file [βœ“]
Thu Aug 23 19:10:57 2018 user.notice simple-adblock [6023]: Creating dnsmasq config [βœ“]
Thu Aug 23 19:10:57 2018 user.notice simple-adblock [6023]: Removing temporary files [βœ“]
Thu Aug 23 19:10:57 2018 daemon.info dnsmasq[6149]: exiting on receipt of SIGTERM
Thu Aug 23 19:10:57 2018 user.notice dnsmasq: DNS rebinding protection is active, will discard upstream RFC1918 responses!
Thu Aug 23 19:10:57 2018 user.notice dnsmasq: Allowing 127.0.0.0/8 responses
Thu Aug 23 19:11:01 2018 user.notice simple-adblock [6023]: Restarting dnsmasq [βœ“]
Thu Aug 23 19:11:01 2018 user.notice simple-adblock [6023]: service is blocking 56084 domains [βœ“]
Thu Aug 23 19:11:01 2018 daemon.crit dnsmasq[6810]: error at line 54643 of /tmp/dnsmasq.d/simple-adblock
Thu Aug 23 19:11:01 2018 daemon.crit dnsmasq[6810]: FAILED to start up
Thu Aug 23 19:11:06 2018 daemon.crit dnsmasq[7008]: error at line 54643 of /tmp/dnsmasq.d/simple-adblock
Thu Aug 23 19:11:06 2018 daemon.crit dnsmasq[7008]: FAILED to start up
Thu Aug 23 19:11:11 2018 daemon.crit dnsmasq[7032]: error at line 54643 of /tmp/dnsmasq.d/simple-adblock
Thu Aug 23 19:11:11 2018 daemon.crit dnsmasq[7032]: FAILED to start up
Thu Aug 23 19:11:16 2018 daemon.crit dnsmasq[7054]: error at line 54643 of /tmp/dnsmasq.d/simple-adblock
Thu Aug 23 19:11:16 2018 daemon.crit dnsmasq[7054]: FAILED to start up
Thu Aug 23 19:11:22 2018 daemon.crit dnsmasq[7062]: error at line 54643 of /tmp/dnsmasq.d/simple-adblock
Thu Aug 23 19:11:22 2018 daemon.crit dnsmasq[7062]: FAILED to start up
Thu Aug 23 19:11:27 2018 daemon.crit dnsmasq[7074]: error at line 54643 of /tmp/dnsmasq.d/simple-adblock
Thu Aug 23 19:11:27 2018 daemon.crit dnsmasq[7074]: FAILED to start up
Thu Aug 23 19:11:27 2018 daemon.info procd: Instance dnsmasq::cfg01411c s in a crash loop 6 crashes, 0 seconds since last crash
sed -n '54643,54645p' < /tmp/dnsmasq.d/simple-adblock
local=/www.turkishαΊ‘irlines.com/
local=/www.tutop.com/
local=/www.tuttosessogratis.org/

Edit: found the culprit list:

https://someonewhocares.org/hosts/hosts

After I removed that everything is bueno!


#124

Upgrade to the most recent package from my repo.


#126

Thanks, great feed. Do you know if it will successfully parse any list from here:

https://filterlists.com/

I'm looking to add the most popular list, EasyList.


#127

No, it doesn't seem to be neither list of domains nor hosts file.

I believe it's designed for the ABP extension to the browser.


#128

Another question, in the readme it states " Please note that these lists have to include either http:// or https:// prefix. Some of the top block lists (both hosts files and domains lists) suitable for routers with at least 8MB RAM are used in the default simple-adblock installation." however many of the blocklists in the default config don't conform to this.

Does it still apply?


#129

Sorry, I'm not sure I understood your question. All the lists in the default config have either http:// or https:// prefix: https://github.com/stangri/openwrt_packages/blob/master/simple-adblock/files/simple-adblock.conf


#130

Hello,

After upgrading my router to OpenWRT 18.06.1 an installing the latest Simple-Adblock, whenever I enable it, its brakes my DNS.

BusyBox v1.28.3 () built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt 18.06.1, r7258-5eb055306f
 -----------------------------------------------------
root@OpenWrt:~# nslookup google.com
Server:         127.0.0.1
Address:        127.0.0.1#53

Name:      google.com
Address 1: 172.217.20.14
Address 2: 2a00:1450:400d:805::200e

root@OpenWrt:~# /etc/init.d/simple-adblock start
Starting simple-adblock 1.6.3-5...
Found existing data file, reusing it βœ“
Restarting dnsmasq βœ“
simple-adblock 1.6.3-5 is blocking 47648 domains βœ“

root@OpenWrt:~# nslookup google.com
;; connection timed out; no servers could be reached

root@OpenWrt:~# nslookup facebook.com
;; connection timed out; no servers could be reached

root@OpenWrt:~# nslookup openwrt.org
;; connection timed out; no servers could be reached

As you can see, at first I can resolv the domain name, but after the script start, I cannot.
How could I debug it? Maybe some similar issue as the previous mentioned earlier.


#131

Which lists are enabled? Does /etc/init.d/dnsmasq restart produce anything of interest?


#133

No, doesnt help


root@OpenWrt:~# /etc/init.d/dnsmasq restart
udhcpc: started, v1.28.3
udhcpc: sending discover
udhcpc: sending select for 192.168.0.120
udhcpc: lease of 192.168.0.120 obtained, lease time 2147483

root@OpenWrt:~# nslookup google.com
;; connection timed out; no servers could be reached
``



I have the default lists enabled. I modified nothing in the config yet.

This is my dnsmasq config. Almost the default, I just added some forward adresses:

config dnsmasq
        option domainneeded '1'
        option localise_queries '1'
        option rebind_protection '1'
        option rebind_localhost '1'
        option local '/lan/'
        option domain 'lan'
        option expandhosts '1'
        option authoritative '1'
        option readethers '1'
        option leasefile '/tmp/dhcp.leases'
        option resolvfile '/tmp/resolv.conf.auto'
        option nonwildcard '1'
        option localservice '1'
        list server '208.67.220.220'                                            
        list server '1.1.1.1'
        list server '8.8.8.8'

#134

Alright, I did some troubleshooting:
After switching on-off all the lists, I found that the following kills my DNS:

#       list blacklist_hosts_url 'http://someonewhocares.org/hosts/hosts'

Shame, I actually like that host file.

How to proceed? Can I fix this? Or how could I help you?


#136

Tracking this post ...

I have exactly the same problem here ... :fearful:

My router: TP-Link WDR3600
OpenWRT: 18.06.1
Simple-Adblock: I'm not sure of the version, I installed it via opkg install simple-adblock

When I start Simple-Adblock, I loose access to DNS and my entire network stops working. It only solves if I turn the router off and on again, since restarting dnsmasq does not have the desired effect.

At this point, I gave up simple-adblock to avoid further problems. :sleepy:


#137

I can suggest you the same, as I did. Disable the lists one-by-one until you found the rogue one.
Since you are there, you could ad couple more lists to widen your protection, from here:

Be aware! If you add to much, your router could ran out of memory. Experiment.


#138

Thanks for your reply @mqmq0

What I did to test the simple-adoblock again on my router, was this:
1 - I installed the package again, it is in the 1.6.3-5 version;
2 - Without starting the same, I removed all the lists, copied them in a notebook;
3 - Start simple-adblock ...
4 - I stopped the simple-adblock ...
5 - I added the lists, one by one and observing the behavior of the router through htop via ssh. I could see that the router's memory usage remained almost unchanged, but when updating each list, CPU usage is 100% for some period.
6 - Upon reaching the "http://someonewhocares.org/hosts/hosts" list, simply the router has stopped working and the network has also stopped. Then, via ssh I stopped the simple-adblock service, removed the list by editing the configuration file and restarted dnsmasq, and everything worked again ... I could not install this list in any way!
7 - After doing this, everything works normally and the CPU usage of the router is low ...

What I could conclude is that if I add all the lists at one time, the high CPU usage causes the router to not be able to process other requests.

Thank you support again - Thanks!

At that time, via htop, my router looks like this:

2018-11-17_19h31_48


#139

I'm guessing that despite my attempts to filter them, some rogue non-ASCII characters still end up in the block-list, screwing up the dnsmasq.

I've pushed an update to my repo which has another filtering for non-ASCII characters on the final block-list, I'd appreciate if people who reported problems could test it.

PS. Also, thanks to @ckuethe, there's now an option to save compressed cache backup in cases where your router has enough flash but unreliable internet connection.


#140

@stangri

Not sure if you fixed it in recent changes, but on 1.6.4-4 I ran into an issue where combining two local lists broke dnsmasq. It turned out to be partly my error as the first list didn't have a trailing whitespace, and when combined with the second list, the last line from the first list merged into the first line from the second list.


#141

Could you please clarify which setting specifically are you talking about?


#142

I have two local "blacklist_domains_url" list entries in my config, A and B. The last entry of list A is bad.site.net. The last entry of list B is ad.cdn.net. List A mistakenly didn't have a trailing whitespace. When running the download function that combines all lists, it concatenated the last entry of list A with the first entry of list B to form bad.site.netad.cdn.net on one line that resulted in breaking dnsmasq when applied. I can't remember the exact resulting entry, but it apparently was something malformed enough to be problematic.

Ideally, there would be check for a trailing whitespace in each list or something that would ensure two different lines never get combined as one.