Adblock support thread

bluudz · December 12, 2019, 2:08am

I have just managed to get the adblock up and running on my Turris Omnia, TurrisOS 4.0.2 5670759 / LuCI branch (git-19.344.19872-31039b6)

In the LuCi adblock is showing as running fine as you can see here:

But when I ssh in and try nslookup the address is still being resolved.

root@turris:~# nslookup doubleclick.net Server: 127.0.0.1 Address: 127.0.0.1#53 Name: doubleclick.net Address 1: 172.217.25.46 Address 2: 2404:6800:4006:802::200e

Also I did try to install chromium without any plugins and ads did appear all over.
What should I try to resolve the issue?

SOLUTION: I have not noticed that the DNS backend has been change from kresd to default, after changing this back to kresd and restarting I seem to be working well!

Thank you for any ideas!

dibdot · December 13, 2019, 8:08am

Hi, thanks for your tests. Could you upload/provide your final blacklist (adb_list.overall) and your whitelist as compressed archive via google drive or similar and PM me the link for further testing?

I've already changed a lot in adblock 4.0 and now I have comparable results with big lists.

fantom-x · December 13, 2019, 12:39pm

You really want the final file, not the intermediate one?

dibdot · December 13, 2019, 12:58pm

I'm fine with with both variants (I can strip the surrounding data if needed).

fantom-x · December 13, 2019, 7:17pm

Ok, I will do that shortly. In the mean time, here is some additional info:

I write the final file to /root/ so that when the router boots up it already has a block list and there is no need to wait until the block list is prepared.
ipq40xx takes an hour
ipq806x takes eleven minutes

Both routers are configured exactly the same, but the slower one has a compressed file system, which slows things down even more. Writing to /tmp/ improves the processing time by a lot of corse.

LondonBridge · December 18, 2019, 4:48am

Hello,

Could anyone help me generate the correct adblock configuration / regex for EU COOKIE NOTICES which are SO annoying and should be included with the default lists.

http://prebake.eu/
and

Ive tried copying configs and settingn the URL to them but I cant work out the valid settings.

erdoukki · December 18, 2019, 11:43am

I have enabled IPv6 from some days and now I get an error with adblock which no more work :
Dec 18 12:37:20 2019 user.err adblock-3.8.14[2751]: dns backend restart with active blocklist failed
I do not know if IPv6 is the problem...
I am testing on the 19.07-RC1 on the MVEBU (EspressoBin Board)

erdoukki · December 18, 2019, 3:52pm

looks like ok with nxdomains, but not ok with null... I am using dnsmasq as DNS...

shelterx · December 22, 2019, 8:46pm

You can't do such a thing with OpenWRT's adblock, it's dns based, so it only blocks on domain level, meaning you can't use URL regex.

Thebroughfamily · December 31, 2019, 8:08am

Adblock running on my R7800 nicely, thank you for the hard work @dibdot...

I'm running it in fairly stock form, dnsmasq and just a few lists ticked... Now about to set up dnscrypt as I believe that works ok with adblock, but theres a few variations listed in the luci repo, is there any more upto date guide than the main help pages item which seems purely command line tutorial based?

Thanks in advance.

SpiceM · January 7, 2020, 6:07pm

installed adblock. seems to work on laptop but does not work with android phone and smart TV which why I wanted to install it in the first place. there are still ads showing on my phone and smart tv e.g. youtube app
how can I make it work on phone and smart tv?

slh · January 7, 2020, 9:24pm

Some devices hardcode their preferred DNS server, bypassing the router and adblock, to intercept their DNS queries you'd need to enable adb_forcedns.

SpiceM · January 8, 2020, 9:23am

Its strange that when I click on the ads on these devices the requested page is blocked so it works but not preventing the ads from showing up (not optimal)
Even with dns leak dns is not leaking when I have the vpn up. Even without vpn no other dns shows up beside isp dns.
Are you certain its these devices own dns query is the reason ads still show up?

accelerate · January 10, 2020, 5:08pm

@dibdot

Does the "DNS query report" feature really need an installation of tcpdump-mini?
I have the normal version of tcpdump installed and it doesn't seem to work.

dibdot · January 10, 2020, 7:59pm

You can't. Adblock blocks only on (sub-) domain level, not on element level.

dibdot · January 10, 2020, 8:03pm

No, it should work just fine with both variants

dibdot · January 10, 2020, 8:07pm

As long as the ad comes from a legit/original side, this behaviour is normal, e.g. sponsored search results on google ...

accelerate · January 10, 2020, 10:06pm

It is activated, but doesn't show any results. No errors in the log.

root@ROUTER:~# cat /etc/config/adblock 

config adblock 'global'
	option adb_basever '3.8'
	option adb_dns 'dnsmasq'
	option adb_fetchutil 'uclient-fetch'
	option adb_trigger 'wan'
	option adb_enabled '1'
	option adb_dnsvariant 'null (IPv4/IPv6)'

config adblock 'extra'
	option adb_debug '0'
	option adb_forcedns '0'
	option adb_maxqueue '4'
	option adb_nice '10'
	option adb_dnsfilereset 'false'
	option adb_backupdir '/tmp'
	option adb_triggerdelay '60'
	option adb_report '1'
	option adb_repdir '/tmp'

config source 'adaway'
	option adb_src 'https://adaway.org/hosts.txt'
	option adb_src_rset '/^127\.0\.0\.1[[:space:]]+([[:alnum:]_-]+\.)+[[:alpha:]]+([[:space:]]|$)/{print tolower(\$2)}'
	option adb_src_desc 'focus on mobile ads, infrequent updates, approx. 400 entries'
	option enabled '1'

config source 'adguard'
	option adb_src 'https://filters.adtidy.org/windows/filters/15.txt'
	option adb_src_rset 'BEGIN{FS=\"[/|^|\r]\"}/^\|\|([[:alnum:]_-]+\.)+[[:alpha:]]+[\/\^\r]+$/{print tolower(\$3)}'
	option adb_src_desc 'combined adguard dns filter list, frequent updates, approx. 17.000 entries'
	option enabled '1'

config source 'bitcoin'
	option adb_src 'https://raw.githubusercontent.com/hoshsadiq/adblock-nocoin-list/master/hosts.txt'
	option adb_src_rset '/^0\.0\.0\.0[[:space:]]+([[:alnum:]_-]+\.)+[[:alpha:]]+([[:space:]]|$)/{print tolower(\$2)}'
	option adb_src_desc 'focus on malicious bitcoin mining sites, infrequent updates, approx. 80 entries'
	option enabled '1'

config source 'disconnect'
	option adb_src 'https://s3.amazonaws.com/lists.disconnect.me/simple_malvertising.txt'
	option adb_src_rset '/^([[:alnum:]_-]+\.)+[[:alpha:]]+([[:space:]]|$)/{print tolower(\$1)}'
	option adb_src_desc 'mozilla driven blocklist, numerous updates on the same day, approx. 4.700 entries'
	option enabled '1'

config source 'dshield'
	option adb_src 'https://www.dshield.org/feeds/suspiciousdomains_Low.txt'
	option adb_src_rset '/^([[:alnum:]_-]+\.)+[[:alpha:]]+([[:space:]]|$)/{print tolower(\$1)}'
	option adb_src_desc 'generic blocklist, daily updates, approx. 3.500 entries'
	option enabled '1'

config source 'hphosts'
	option adb_src 'https://hosts-file.net/ad_servers.txt'
	option adb_src_rset '/^127\.0\.0\.1[[:space:]]+([[:alnum:]_-]+\.)+[[:alpha:]]+([[:space:]]|\$)+/{print tolower(\$2)}'
	option adb_src_desc 'broad blocklist, monthly updates, approx. 19.200 entries'
	option enabled '1'

config source 'malware'
	option adb_src 'https://mirror.espoch.edu.ec/malwaredomains/justdomains'
	option adb_src_rset '/^([[:alnum:]_-]+\.)+[[:alpha:]]+([[:space:]]|$)/{print tolower(\$1)}'
	option adb_src_desc 'broad blocklist, daily updates, approx. 18.300 entries'
	option enabled '1'

config source 'malwarelist'
	option adb_src 'http://www.malwaredomainlist.com/hostslist/hosts.txt'
	option adb_src_rset '/^127\.0\.0\.1[[:space:]]+([[:alnum:]_-]+\.)+[[:alpha:]]+([[:space:]]|$)/{print tolower(\$2)}'
	option adb_src_desc 'focus on malware, daily updates, approx. 1.200 entries'
	option enabled '1'

config source 'openphish'
	option adb_src 'https://openphish.com/feed.txt'
	option adb_src_rset 'BEGIN{FS=\"/\"}/^http[s]?:\/\/([[:alnum:]_-]+\.)+[[:alpha:]]+(\/|$)/{print tolower(\$3)}'
	option adb_src_desc 'focus on phishing, numerous updates on the same day, approx. 2.400 entries'
	option enabled '1'

config source 'reg_cn'
	option adb_src 'https://easylist-downloads.adblockplus.org/easylistchina+easylist.txt'
	option adb_src_rset 'BEGIN{FS=\"[|^]\"}/^\|\|([[:alnum:]_-]+\.)+[[:alpha:]]+\^("\\\$third-party")?$/{print tolower(\$3)}'
	option adb_src_desc 'focus on chinese ads plus generic easylist additions, daily updates, approx. 11.700 entries'
	option enabled '1'

config source 'reg_cz'
	option adb_src 'https://raw.githubusercontent.com/qxstyles/turris-hole-czech-block-list/master/turris-hole-czech-block-list'
	option adb_src_rset '/^([[:alnum:]_-]+\.)+[[:alpha:]]+([[:space:]]|$)/{print tolower(\$1)}'
	option adb_src_desc 'focus on czech ads maintained by Turris Omnia Users, infrequent updates, approx. 100 entries'
	option enabled '1'

config source 'reg_de'
	option adb_src 'https://easylist-downloads.adblockplus.org/easylistgermany+easylist.txt'
	option adb_src_rset 'BEGIN{FS=\"[|^]\"}/^\|\|([[:alnum:]_-]+\.)+[[:alpha:]]+\^("\\\$third-party")?$/{print tolower(\$3)}'
	option adb_src_desc 'focus on german ads plus generic easylist additions, daily updates, approx. 9.200 entries'
	option enabled '1'

config source 'reg_id'
	option adb_src 'https://easylist-downloads.adblockplus.org/abpindo+easylist.txt'
	option adb_src_rset 'BEGIN{FS=\"[|^]\"}/^\|\|([[:alnum:]_-]+\.)+[[:alpha:]]+\^("\\\$third-party")?$/{print tolower(\$3)}'
	option adb_src_desc 'focus on indonesian ads plus generic easylist additions, weekly updates, approx. 9.600 entries'
	option enabled '1'

config source 'reg_nl'
	option adb_src 'https://easylist-downloads.adblockplus.org/easylistdutch+easylist.txt'
	option adb_src_rset 'BEGIN{FS=\"[|^]\"}/^\|\|([[:alnum:]_-]+\.)+[[:alpha:]]+\^("\\\$third-party")?$/{print tolower(\$3)}'
	option adb_src_desc 'focus on dutch ads plus generic easylist additions, weekly updates, approx. 9.400 entries'
	option enabled '1'

config source 'reg_pl'
	option adb_src 'http://adblocklist.org/adblock-pxf-polish.txt'
	option adb_src_rset 'BEGIN{FS=\"[|^]\"}/^\|\|([[:alnum:]_-]+\.)+[[:alpha:]]+\^("\\\$third-party")?$/{print tolower(\$3)}'
	option adb_src_desc 'focus on polish ads, daily updates, approx. 90 entries'
	option enabled '1'

config source 'reg_ro'
	option adb_src 'https://easylist-downloads.adblockplus.org/rolist+easylist.txt'
	option adb_src_rset 'BEGIN{FS=\"[|^]\"}/^\|\|([[:alnum:]_-]+\.)+[[:alpha:]]+\^("\\\$third-party")?$/{print tolower(\$3)}'
	option adb_src_desc 'focus on romanian ads plus generic easylist additions, weekly updates, approx. 9.400 entries'
	option enabled '1'

config source 'reg_ru'
	option adb_src 'https://easylist-downloads.adblockplus.org/ruadlist+easylist.txt'
	option adb_src_rset 'BEGIN{FS=\"[|^]\"}/^\|\|([[:alnum:]_-]+\.)+[[:alpha:]]+\^("\\\$third-party")?$/{print tolower(\$3)}'
	option adb_src_desc 'focus on russian ads plus generic easylist additions, weekly updates, approx. 14.500 entries'
	option enabled '1'

config source 'shalla'
	option adb_src 'http://www.shallalist.de/Downloads/shallalist.tar.gz'
	option adb_src_rset '/^([[:alnum:]_-]+\.)+[[:alpha:]]+([[:space:]]|$)/{print tolower(\$1)}'
	option adb_src_desc 'huge blocklist archive subdivided in different categories, daily updates. Check http://www.shallalist.de/categories.html for more categories'
	option enabled '1'
	list adb_src_cat 'adv'
	list adb_src_cat 'costtraps'
	list adb_src_cat 'spyware'
	list adb_src_cat 'tracker'
	list adb_src_cat 'warez'
	list adb_src_cat 'anonvpn'
	list adb_src_cat 'dynamic'
	list adb_src_cat 'redirector'
	list adb_src_cat 'remotecontrol'
	list adb_src_cat 'urlshortener'
	list adb_src_cat 'violence'

config source 'spam404'
	option adb_src 'https://raw.githubusercontent.com/Dawsey21/Lists/master/main-blacklist.txt'
	option adb_src_rset '/^([[:alnum:]_-]+\.)+[[:alpha:]]+([[:space:]]|$)+/{print tolower(\$1)}'
	option adb_src_desc 'generic blocklist, infrequent updates, approx. 6.000 entries'
	option enabled '1'

config source 'sysctl'
	option adb_src 'http://sysctl.org/cameleon/hosts'
	option adb_src_rset '/^127\.0\.0\.1[[:space:]]+([[:alnum:]_-]+\.)+[[:alpha:]]+([[:space:]]|$)/{print tolower(\$2)}'
	option adb_src_desc 'broad blocklist, weekly updates, approx. 16.500 entries'
	option enabled '1'

config source 'ut_capitole'
	option adb_src 'https://dsi.ut-capitole.fr/blacklists/download/blacklists.tar.gz'
	option adb_src_rset '/^([[:alnum:]_-]+\.)+[[:alpha:]]+([[:space:]]|$)/{print tolower(\$1)}'
	option adb_src_desc 'huge blocklist archive subdivided in different categories, daily updates. Check https://dsi.ut-capitole.fr/blacklists/index_en.php for more categories'
	option enabled '1'
	list adb_src_cat 'publicite'
	list adb_src_cat 'cryptojacking'
	list adb_src_cat 'ddos'
	list adb_src_cat 'malware'
	list adb_src_cat 'phishing'
	list adb_src_cat 'warez'
	list adb_src_cat 'bitcoin'
	list adb_src_cat 'download'
	list adb_src_cat 'marketingware'
	list adb_src_cat 'reaffected'
	list adb_src_cat 'redirector'
	list adb_src_cat 'shortener'

config source 'whocares'
	option adb_src 'https://someonewhocares.org/hosts/hosts'
	option adb_src_rset '/^127\.0\.0\.1[[:space:]]+([[:alnum:]_-]+\.)+[[:alpha:]]+([[:space:]]|$)/{print tolower(\$2)}'
	option adb_src_desc 'broad blocklist, weekly updates, approx. 10.000 entries'
	option enabled '1'

config source 'winspy'
	option adb_src 'https://raw.githubusercontent.com/crazy-max/WindowsSpyBlocker/master/data/hosts/spy.txt'
	option adb_src_rset '/^0\.0\.0\.0[[:space:]]+([[:alnum:]_-]+\.)+[[:alpha:]]+([[:space:]]|$)/{print tolower(\$2)}'
	option adb_src_desc 'focus on windows spy & telemetry domains, infrequent updates, approx. 300 entries'
	option enabled '1'

config source 'winhelp'
	option adb_src 'http://winhelp2002.mvps.org/hosts.txt'
	option adb_src_rset '/^0\.0\.0\.0[[:space:]]+([[:alnum:]_-]+\.)+[[:alpha:]]+([[:space:]]|$)/{print tolower(\$2)}'
	option adb_src_desc 'broad blocklist, infrequent updates, approx. 13.000 entries'
	option enabled '1'

config source 'yoyo'
	option adb_src 'https://pgl.yoyo.org/adservers/serverlist.php?hostformat=nohtml&showintro=0&mimetype=plaintext'
	option adb_src_rset '/^([[:alnum:]_-]+\.)+[[:alpha:]]+([[:space:]]|$)/{print tolower(\$1)}'
	option adb_src_desc 'focus on ad related domains, weekly updates, approx. 2.400 entries'
	option enabled '1'

root@ROUTER:~# opkg list-installed | grep tcpdump
tcpdump - 4.9.3-1

SpiceM · January 10, 2020, 11:00pm

it blocks the ads (youtube for example and other streaming apps) on pc but they show up (the same ads blocked in the pc) in smartphone and TV. I need to block them on the smartphone and the TV otherwise no need for adblock at router level since I can just use adblock extension on pc

dibdot · January 11, 2020, 5:01am

You can't block ads reliable from youtube, cause they're coming from a more or less random xxx.youtube.com sub-domain (and of course you can't block youtube.com ;-)). Most probably on your PC are running additional plugins like ublock etc. which are blocking on element level.