Adblock-fast: ad-blocking service for dnsmasq, smartdns and unbound

In this condition, what are the outputs of:

service adblock-fast info
cat /dev/shm/adblock-fast-status.json

Also, if you want to try 1.0.0-9 from my repo to see if the problem persists, that'd be nice.

root@OpenWrt:~# service adblock-fast info

{
"adblock-fast": {
"triggers": [
[
"interface.*",
[
"if",
[
"eq",
"interface",
"wan"
],
[
"run_script",
"/etc/init.d/adblock-fast",
"start"
]
],
1000
],
[
"config.change",
[
"if",
[
"eq",
"package",
"adblock-fast"
],
[
"run_script",
"/etc/init.d/adblock-fast",
"reload"
]
],
1000
]
]
}
}

root@OpenWrt:~# cat /dev/shm/adblock-fast-status.json

{ "data": { "version": "1.0.0-7", "status": "statusSuccess", "message": "", "error": "", "stats": "adblock-fast 1.0.0-7 is blocking 379493 domains (with dnsmasq.servers)", "reload": "1 0 10 \t\t\t\t\t\t https://cdn.jsdelivr.net/gh/StevenBlack/hosts/hosts https://raw.githubusercontent.com/AdguardTeam/cname-trackers/master/data/combined_disguised_trackers_justdomains.txt https://big.oisd.nl/ https://cdn.jsdelivr.net/gh/bongochong/CombinedPrivacyBlockLists/NoFormatting/cpbl-ctld.txt http://sysctl.org/cameleon/hosts https://cdn.jsdelivr.net/gh/kboghdady/youTube_ads_4_pi-hole/black.list https://raw.githubusercontent.com/AdguardTeam/cname-trackers/master/data/combined_disguised_clickthroughs_justdomains.txt https://someonewhocares.org/hosts/hosts https://winhelp2002.mvps.org/hosts.txt https://adaway.org/hosts.txt https://raw.githubusercontent.com/AdguardTeam/cname-trackers/master/data/combined_disguised_ads_justdomains.txt https://raw.githubusercontent.com/AdguardTeam/cname-trackers/master/data/combined_disguised_microsites_justdomains.txt https://pgl.yoyo.org/as/serverlist.php?hostformat=hosts&showintro=1&mimetype=plaintext https://cdn.jsdelivr.net/gh/hoshsadiq/adblock-nocoin-list/hosts.txt dnsmasq.servers \t\t\t\t\t\t0 https://cdn.jsdelivr.net/gh/openwrt/packages/net/adblock-fast/files/adblock-fast.conf.update \t\t\t\t\t\t 30000000 3", "restart": "0 /etc 1 \t\t\t\t\t\t53 853" } }

I tried, removed the older version first but it failed to install 1.0.0-9,

Problem is, there is no package to uninstall now.

So, Im kinda stuck with that now lol.

I disabled a feed which triggered a reload and saw this error in the log

Fri Nov  3 04:38:28 2023 user.notice adblock-fast [11246]: Reloading dnsmasq [✓]
Fri Nov  3 04:38:29 2023 user.notice adblock-fast [11246]: service is blocking 375769 domains (with dnsmasq.servers) [✓]
Fri Nov  3 04:38:29 2023 user.notice adblock-fast [11246]: ERROR: failed to process allow-list!

After removing cdn.jsdelivr.net entry from the allow-list, the status is now active again

Can you please elaborate on what did you disable and how?

If you can reproduce problems with 1.0.0-9, please post your /etc/config/adblock-fast files and I'll try to reproduce.

Hi I am new user to adblock-fast and I am having an issue with 1.0.0-7, namely the whitelist. Now I found that I cannot search on the HomeDepot site (disabling adblock makes it work). Figured I would start allowing a few things. I used the CLI command to find the domains in the block list that relate to HD and was going to add them to the "Allowed Domains". As a test a added one and get an "ERROR: failed to process allow-list!" (tried a bunch of different formats). I deleted the HD entry leaving the default "cdn.jsdelivr.net" to revert back but the error remains. Removing "cdn.jsdelivr.net" allows the application to work again but does not solve my problem.
Any thoughts as to why the whitelist is not working for me?
BTW where is the log file located?
Thx

  1. in 1.0.0-7 I was able to get the status from Stopped to Active by disabling this feed and hitting Save & Apply

  2. this morning I stopped the adblock-fast service, downloaded 1.0.0-9 packages from https://github.com/stangri/repo.openwrt.melmac.net and updated them using LuCI

  • the adblock-fast_1.0.0-9_all.ipk update throwed following error
Upgrading adblock-fast on root from 1.0.0-7 to 1.0.0-9...
Stopping service and removing rc.d symlink for adblock-fast
Configuring adblock-fast.

Command failed: ubus call service delete { "name": "adblock-fast" } (Not found)
Command failed: ubus call service delete { "name": "adblock-fast" } (Not found)
sed: couldn't write 33 items to stdout: Broken pipe
sed: couldn't write 17 items to stdout: Broken pipe
sed: couldn't write 18 items to stdout: Broken pipe
sed: couldn't write 25 items to stdout: Broken pipe
sed: couldn't write 14 items to stdout: Broken pipe
Collected errors:
 * resolve_conffiles: Existing conffile /etc/config/adblock-fast is different from the conffile in the new package. The new conffile will be placed at /etc/config/adblock-fast-opkg.

The luci-app-adblock-fast_1.0.0-9_all.ipk update went ok

After installation the status in LuCI looked ok.

I added cdn.jsdelivr.net again to allowed domains, hit Save & Apply but the error turned up again.

Sun Nov 5 06:37:23 2023 user.notice adblock-fast [562]: Reloading dnsmasq [✓]
Sun Nov 5 06:37:24 2023 user.notice adblock-fast [562]: service is blocking 369591 domains (with dnsmasq.servers) [✓]
Sun Nov 5 06:37:24 2023 user.notice adblock-fast [562]: ERROR: failed to process allow-list!

/etc/config/adblock-fast
config adblock-fast 'config'
	option enabled '1'
	option allow_non_ascii '0'
	option canary_domains_icloud '0'
	option canary_domains_mozilla '0'
	option compressed_cache '0'
	option compressed_cache_dir '/etc'
	option config_update_enabled '0'
	option config_update_url 'https://cdn.jsdelivr.net/gh/openwrt/packages/net/adblock-fast/files/adblock-fast.conf.update'
	option curl_max_file_size '30000000'
	option curl_retry '3'
	option debug '0'
	option dns 'dnsmasq.servers'
	option download_timeout '10'
	option force_dns '1'
	list force_dns_port '53'
	list force_dns_port '853'
	option parallel_downloads '1'
	option pause_timeout '20'
	option procd_trigger_wan6 '0'
	option procd_boot_wan_timeout '60'
	option verbosity '2'
	option dnsmasq_instance '*'
	list allowed_domain 'cdn.jsdelivr.net'

config file_url
	option url 'https://cdn.jsdelivr.net/gh/StevenBlack/hosts/hosts'
	option size '6770929'
	option action 'block'

config file_url
	option url 'https://raw.githubusercontent.com/AdguardTeam/cname-trackers/master/data/combined_disguised_trackers_justdomains.txt'
	option size '6241707'
	option action 'block'

config file_url
	option url 'https://big.oisd.nl/'
	option size '6163363'
	option action 'block'

config file_url
	option url 'https://cdn.jsdelivr.net/gh/bongochong/CombinedPrivacyBlockLists/NoFormatting/cpbl-ctld.txt'
	option size '2608152'
	option action 'block'

config file_url
	option url 'http://sysctl.org/cameleon/hosts'
	option size '638545'
	option action 'block'

config file_url
	option url 'https://cdn.jsdelivr.net/gh/kboghdady/youTube_ads_4_pi-hole/black.list'
	option size '553006'
	option action 'block'

config file_url
	option url 'https://raw.githubusercontent.com/AdguardTeam/cname-trackers/master/data/combined_disguised_clickthroughs_justdomains.txt'
	option size '362170'
	option action 'block'

config file_url
	option url 'https://someonewhocares.org/hosts/hosts'
	option size '347410'
	option action 'block'

config file_url
	option url 'https://winhelp2002.mvps.org/hosts.txt'
	option size '334861'
	option action 'block'

config file_url
	option url 'https://adaway.org/hosts.txt'
	option size '243454'
	option action 'block'

config file_url
	option url 'https://raw.githubusercontent.com/AdguardTeam/cname-trackers/master/data/combined_disguised_ads_justdomains.txt'
	option size '222595'
	option action 'block'

config file_url
	option url 'https://raw.githubusercontent.com/AdguardTeam/cname-trackers/master/data/combined_disguised_microsites_justdomains.txt'
	option size '123275'
	option action 'block'

config file_url
	option url 'https://pgl.yoyo.org/as/serverlist.php?hostformat=hosts&showintro=1&mimetype=plaintext'
	option size '99588'
	option action 'block'

config file_url
	option url 'https://cdn.jsdelivr.net/gh/hoshsadiq/adblock-nocoin-list/hosts.txt'
	option size '11149'
	option action 'block'

**Edit** really strange: I just found out that with Enable Debugging checked that the error does not show up

@stangri Various luci-app-adblock-fast and luci-app-https-dns-proxy string were added/updated last week on GitHub but on Weblate only the English (Developer) got them, so other locales have outdated strings.

@ed8 What's the output of ubus call system board? Which version/flavour of OpenWrt it is? If things work if you disable some URLs, you're probably running out of memory.

I just pushed adblock-fast 1.0.0-10 and luci-app-adblock-fast 1.0.0-10 to my repo, I'd appreciate testing by people with older routers/smaller RAM. This version checks available RAM and compares it to the file sizes of enabled lists before proceeding to download.

version is OpenWrt v23.05 running on RP CM4 with 2Gb RAM

ubus call system board
{
	"kernel": "5.15.134",
	"hostname": "router",
	"system": "ARMv8 Processor rev 3",
	"model": "Raspberry Pi Compute Module 4 Rev 1.1",
	"board_name": "raspberrypi,4-compute-module",
	"rootfs_type": "squashfs",
	"release": {
		"distribution": "OpenWrt",
		"version": "23.05.0",
		"revision": "r23497-6637af95aa",
		"target": "bcm27xx/bcm2711",
		"description": "OpenWrt 23.05.0 r23497-6637af95aa"
	}
}

I am not running out of memory

root@router:~# free -m
              total        used        free      shared  buff/cache   available
Mem:        1901344       95100     1746152       13640       60092     1740884
Swap:             0           0           0

I can only workaround the "ERROR: failed to process allow-list!" when I check "Enable Debugging"

I suspect it's an issue with the busybox-included sed. What's the output of ls -la $(which sed)?

Does it help if you do opkg update; opkg install sed and then service adblock-fast dl?

<pre>root@router:~# ls -la $(which sed)
lrwxrwxrwx    1 root     root            20 Oct 31 18:02 <font color="#2AA1B3"><b>/bin/sed</b></font> -&gt; <font color="#26A269"><b>/usr/libexec/sed-gnu</b></font>
</pre>

After
root@router:~# opkg update; opkg install sed
and service adblock-fast dl
I still get the error
image

If I enable debugging it also works for me, the Force Router DNS flag doesnt work as intended if I do that though.

I tried to block a custom domain (itch.io) and it didn't work. This is the config:

image

And this is the check:

1 Like

Thanks for report. Depending on remote block-lists, the prepared combined file not always ended with a new line which prevented first in the individual domains list from being blocked.

Was fixed in adblock-fast 1.0.1-2 in my repo, I'll send PR/merge changes in the OpenWrt repo within a few days.

I've got a problem, when using AdBlockPlus lists: It filters the complete host/domain even when the list wants to filter only some few URLs from this host.

Of course a DNS based filter can only filter the whole host, it can't block some specific URLs. Therefore I suggest to filter only hosts, that are completely blocked in the AdBlockPlus list. Partially blocked hosts should be ignored.

An example: The reg_de (https://easylist.to/easylistgermany/easylistgermany.txt) list contains several entries blocking parts of the amazon.de site. Adblock-fast blocks whole amazon.de, what is certainly not what the user wants.

As a workaround I'm using this filter to make a domain list from the reg_de list:
This is just a quick-and dirty solution, I have only a basic understanding of the AdBlockPlus format.

curl -sS --fail -o /tmp/reg_de.txt https://easylist.to/easylistgermany/easylistgermany.txt || exit
sed -E -n 's/^\|\|([a-zA-Z0-9.-]+)([/^](\$((popup|subdocument|third-party),?)*)?)?$/\1/p' /tmp/reg_de.txt > /etc/config/adlists/reg_de.txt
1 Like

It should only do that if another list contains the higher level domain to optimize the final block-list.