Adblock-fast: ad-blocking service for dnsmasq, smartdns and unbound

I have the first list (Hagezi Pro, the largest) selected. Every time I start or redownload from Luci the browser shows a never-ending "hourglass" popup. If I F5 it will load the page fine and shows 191732 domains blocked using dnsmasq.servers.

I have the recommended tools (gawk, grep, sed & sort) installed and the router is a quad-core with 1GiB memory, more than capable of handling it.

I can't access some sites, e.g. duckduckgo.com via browser anymore when activating Adblock-fast.

No matter what list I choose, Hagezi Pro, Hagezi Normal, AdguardTeam - CNAME Trackers, OISD - Big.

I am also able to ping duckduckgo.com from the devices.
But via browser I only get NS_ERROR_CONNECTION_REFUSED

When stopping the service it is working again.

Probably not an Adblock-fast application issue, but can someone explain?

(Also allowlisting didn't make the site accessible again so far)

Version 1.1.2-r20 - Active.
Blocking 166717 domains (with dnsmasq.nftset).

Why use nftset? It’s probably blocking an IP shared with the site you are trying to reach.

2 Likes

Thanks, switching to dnsmasq.addnhosts fixed it.
Just set it up, went through the docs and thought the DoH feature of nftset sounded good to start with.
Didn't realize that IP address based blocking is completely different from the other methods and not what I wanted.

Are you using the "dnsmasq additional hosts" option on DNS Service? That should load fast even with largest lists.

Hello.
I just installed adblock-fast and its luci-app on openwrt 23.05 but i'm getting an error on luci.


What should I do?

Edit: I used https://firmware-selector.openwrt.org, searched for my router and just add 'luci-app-adblock-fast', download the sysupdate and flash. Still the error shows on System->Status and on Services->Adblock-fast.

I have been using adblock-fast for a long time, periodically adding new websites to the allow list in the Web UI. I have roughly 80 entries now

Recently I have noticed that I get a "Failed to process allow-list!" error in the Luci UI, with no indication of what went wrong in the log. I tried enabling debug log, but I can't really make sense of the output (I don't know what to look for... searching for "allow" "error" and similar doesn't really help me find the offending line, and the file is too long to parse line by line)

What's the best way to identify what is triggering the "Failed to process allow-list!" message?

Adding my config file


config adblock-fast 'config'
	option enabled '1'
	option allow_non_ascii '0'
	option canary_domains_icloud '0'
	option canary_domains_mozilla '0'
	option compressed_cache '0'
	option compressed_cache_dir '/etc'
	option config_update_enabled '1'
	option config_update_url 'https://cdn.jsdelivr.net/gh/openwrt/packages/net/adblock-fast/files/adblock-fast.config.update'
	option curl_max_file_size '30000000'
	option curl_retry '3'
	option debug '1'
	option dns 'dnsmasq.servers'
	option download_timeout '10'
	option force_dns '1'
	list force_dns_port '53'
	list force_dns_port '853'
	option parallel_downloads '1'
	option pause_timeout '20'
	option procd_trigger_wan6 '0'
	option procd_boot_delay '0'
	option procd_boot_wan_timeout '60'
	option verbosity '2'
	option dnsmasq_instance '*'
	list allowed_domain 'cdn.jsdelivr.net'
	list allowed_domain 'aka.ms'
	list allowed_domain 'api.target.com'
	list allowed_domain 'app.adjust.com'
	list allowed_domain 'builtwith.com'
	list allowed_domain 'cdn.ampproject.org'
	list allowed_domain 'click.linksynergy.com'
	list allowed_domain 'dev.virtualearth.net'
	list allowed_domain 'display.ugc.bazaarvoice.com'
	list allowed_domain 'download.cnet.com'
	list allowed_domain 'ecn.dev.virtualearth.net'
	list allowed_domain 'evite.com'
	list allowed_domain 'f.wishabi.net'
	list allowed_domain 'gfwsl.geforce.com'
	list allowed_domain 'm.hotmail.com'
	list allowed_domain 'media.licdn.com'
	list allowed_domain 'officeclient.microsoft.com'
	list allowed_domain 'platform.linkedin.com'
	list allowed_domain 'pleasanthillgrain.com'
	list allowed_domain 'redsky.target.com'
	list allowed_domain 'rover.ebay.com'
	list allowed_domain 's.click.aliexpress.com'
	list allowed_domain 'static.licdn.com'
	list allowed_domain 't0.ssl.ak.dynamic.tiles.virtualearth.net'
	list allowed_domain 't0.ssl.ak.tiles.virtualearth.net'
	list allowed_domain 'taglocker.target.com'
	list allowed_domain 'thedailybeast.com'
	list allowed_domain 'www.awin1.com'
	list allowed_domain 'www.crunchbase.com'
	list allowed_domain 'www.emergency24.com'
	list allowed_domain 'www.everydayhealth.com'
	list allowed_domain 'www.evite.com'
	list allowed_domain 'www.gearbest.com'
	list allowed_domain 'www.liveleak.com'
	list allowed_domain 'www.maxim.com'
	list allowed_domain 'www.oglaf.com'
	list allowed_domain 'www.pinterest.com'
	list allowed_domain 'www.redstate.com'
	list allowed_domain 'www.thedailybeast.com'
	list allowed_domain 'www.tucows.com'
	list allowed_domain 'www.ursecure.com'
	list allowed_domain 'www.videolan.org'
	list allowed_domain 'www.washingtonexaminer.com'
	list allowed_domain 'fls-na.amazon.com'
	list allowed_domain 'skype.com'
	list allowed_domain 'www.dpbolvw.net'
	list allowed_domain 'cj.dotomi.com'
	list allowed_domain 'www.emjcd.com'
	list allowed_domain 'www.ojrq.net'
	list allowed_domain 'js-agent.newrelic.com'
	list allowed_domain 'res.cloudinary.com'
	list allowed_domain 'mcphee.com'
	list allowed_domain 'www.jdoqocy.com'
	list allowed_domain 'thedailywtf.com'
	list allowed_domain 'www.kqzyfj.com'
	list allowed_domain 'japanesechefsknife.com'
	list allowed_domain 'sony.co.uk'
	list allowed_domain 'www.sony.co.uk'
	list allowed_domain 'www.rawstory.com'
	list allowed_domain 'best.aliexpress.com'
	list allowed_domain 'spclient.wg.spotify.com'
	list allowed_domain 'www.redgifs.com'
	list allowed_domain 'sdk.split.io'
	list allowed_domain 'device-metrics-us-2.amazon.com'
	list allowed_domain 's3-iad-2.cf.dash.row.aiv-cdn.net'
	list allowed_domain 'logfiles.zoom.us'
	list allowed_domain 'www.awxcdn.com'
	list allowed_domain 'cdn.amplitude.com'
	list allowed_domain 'static.cdn.responsys.net'
	list allowed_domain 'www.breitbart.com'
	list allowed_domain 'bc0a.com'
	list allowed_domain 'cdn.bc0a.com'
	list allowed_domain 'marvel-b1-cdn.bc0a.com'
	list allowed_domain 'data.digital.costco.com'
	list allowed_domain 'cdn.lr-in-prod.com'
	list allowed_domain 'lr-in-prod.com'
	list allowed_domain 'r.lr-in-prod.com'
	list allowed_domain 'assets.adobedtm.com'
	list allowed_domain 'bbc.com'
	list allowed_domain 'bbc.co.uk'

config file_url
	option url 'https://cdn.jsdelivr.net/gh/StevenBlack/hosts/hosts'
	option size '4105296'
	option action 'block'
	option name 'StevenBlack - Unified hosts'

config file_url
	option url 'https://raw.githubusercontent.com/AdguardTeam/cname-trackers/master/data/combined_disguised_trackers_justdomains.txt'
	option size '6241707'
	option action 'block'
	option enabled '0'
	option name 'AdguardTeam - CNAME Trackers'

config file_url
	option url 'https://big.oisd.nl/'
	option size '6163363'
	option action 'block'
	option enabled '0'
	option name 'OISD - Big'

config file_url
	option url 'https://cdn.jsdelivr.net/gh/bongochong/CombinedPrivacyBlockLists/NoFormatting/cpbl-ctld.txt'
	option size '2608152'
	option action 'block'
	option enabled '0'
	option name 'Bongochong - Combined Privacy Block Lists (TLD Optimized)'

config file_url
	option size '638545'
	option action 'block'
	option enabled '0'
	option name 'config'

config file_url
	option url 'https://cdn.jsdelivr.net/gh/kboghdady/youTube_ads_4_pi-hole/black.list'
	option size '553006'
	option action 'block'
	option enabled '0'
	option name 'Kboghdady - YouTube Ads DNS'

config file_url
	option url 'https://raw.githubusercontent.com/AdguardTeam/cname-trackers/master/data/combined_disguised_clickthroughs_justdomains.txt'
	option size '362170'
	option action 'block'
	option enabled '0'
	option name 'AdguardTeam - CNAME Clickthroughs'

config file_url
	option url 'https://someonewhocares.org/hosts/hosts'
	option size '347410'
	option action 'block'
	option enabled '0'
	option name 'SomeoneWhoCares - Hosts'

config file_url
	option url 'https://winhelp2002.mvps.org/hosts.txt'
	option size '334861'
	option action 'block'
	option enabled '0'
	option name 'WinHelp2002 MVPS - Hosts'

config file_url
	option url 'https://adaway.org/hosts.txt'
	option size '243454'
	option action 'block'
	option enabled '0'
	option name 'AdAway - Hosts'

config file_url
	option url 'https://raw.githubusercontent.com/AdguardTeam/cname-trackers/master/data/combined_disguised_ads_justdomains.txt'
	option size '222595'
	option action 'block'
	option enabled '0'
	option name 'AdguardTeam - CNAME Ads'

config file_url
	option url 'https://raw.githubusercontent.com/AdguardTeam/cname-trackers/master/data/combined_disguised_microsites_justdomains.txt'
	option size '123275'
	option action 'block'
	option enabled '0'
	option name 'AdguardTeam - CNAME Microsites'

config file_url
	option url 'https://pgl.yoyo.org/as/serverlist.php?hostformat=hosts&showintro=1&mimetype=plaintext'
	option size '99588'
	option action 'block'
	option enabled '0'
	option name 'Yoyo.org - Hosts'

config file_url
	option url 'https://cdn.jsdelivr.net/gh/hoshsadiq/adblock-nocoin-list/hosts.txt'
	option size '11149'
	option action 'block'
	option enabled '0'
	option name 'Hoshsadiq - NoCoin Adblock List'

config file_url
	option action 'block'
	option url 'https://hostfiles.frogeye.fr/firstparty-trackers-hosts.txt'
	option size '1079035'

config file_url
	option enabled '0'
	option action 'block'
	option url 'https://raw.githubusercontent.com/hagezi/dns-blocklists/main/dnsmasq/pro.txt'
	option size '5493470'

Anyone? Is there a better place to ask this question?

I think it's all about the syntax of your file.
Take and download any of the preset addresses of blocking files and study the syntax in it and bring your file to the same parameters and everything should work.

Thanks, but at the moment I entered all the whitelisted entities in the Luci UI, and there is only one format possible (xxx.domain.tld or domain.tld).

I don't know which entry is causing problems. I guess I could use a sort of binary search (each time removing half) to find the problem, but it's time consuming.

There has to be a way to use the debug log to find the offending entry. After all, if the Luci UI reports it, it must come from the parsing steps

The debug log is output to a remote computer as an option, maybe there is something there, but in all the files that are provided as possible lists by default everything is output as
domain.com
but not
www.domain.com
https://www.domain.com
etc.

But I may be wrong)). Probably it is better to wait for the author's response.

When you enable debug, it creates the /tmp/adblock-fast.log file locally on the router

I followed the readme in choosing the format of the allowed domains. Please note that I'm not using an allow-list file, just added every entry manually in the Luci UI. I followed this:

If you specify google.com as a domain to be allowed, you will have access to google.com, www.google.com, analytics.google.com, but not fake domains like email-google.com or drive.google.com.verify.signin.normandeassociation.com for example. If you only want to allow www.google.com while blocking all other google.com subdomains, just specify www.google.com as domain to be allowed.

As you can see, domain.org and www.domain.org are both valid options, the former allows everything on domain.org, the latter only explicitly allows www.domain.org (so, for example, foo.domain.org would still be blocked if present in a block list)

It should work, but it doesn't and I can't figure out how to troubleshoot the offending entry

Thanks everyone, for clarifying dnsmasq.nftset for me.
The log info helped me understand what is transpiring.

I ran a custom firmware for a while that was incompatible with adblock-fast.
I have reconfigured and back to running adblock-fast.
I tried dnsmasq.nftset again and again got blocked to sites that shouldn't block.
Enabling logs quickly told me this was the wrong option for me.

My current setup has HTTPS DNS Proxy redirecting browser lookups.
dnsmasq.server configuration is working great in this setup.

Hello,

I'm working on small optimization when processing domains list format files and looking for support from people who are willing to test the code on different platforms and share the results. Tests should be made with Hagezi - Pro (domains) list which is big enough to see the difference in times between the current version of adblock-fast (1.1.3-r11) and the modified one which can be found in my branch adblock-fast_opt_domains This is the related PR in the official repo. https://github.com/stangri/adblock-fast/pull/1

Thank you in advance for your cooperation.

3 Likes

Just wanted to point out that any cooperation with @justops1337 efforts will be greatly appreciated and hugely beneficial to a lot of users. @justops1337 has been submitting various bug-fixes and performance improvements to adblock-fast and we both would appreciate a variety of users testing this fix on different platforms.

1 Like

Reposting this, hoping for more luck. I can't believe that there's no way to debug an error message like that. Can someone please point me in the right direction? Any way to contact the developer?

Hi, just wanna say thanks to @stangri for making adblock-fast, pbr and http-proxy packages.

They are very useful, easy to configure and integrates with each other perfectly. :+1:

4 Likes

Yeah, he is da best :slight_smile:

1 Like

Please provide some additional information about the issue.

  • Which is the version of your adblock-fast?
  • Which is the version of your luci-app-adblock-fast?

I took your configuration file from the previous post and it works fine.

Is this intentional or it's just a typo in your configuration?

config file_url
        option size '638545'
        option action 'block'
        option enabled '0'
        option name 'config'

Thanks so much for looking into this. 1.1.2-20 and 1.1.2 respectively, the newest versions available

As for that weird config file, I'm positive that I never edited it, it must have been something it picked up in an old version. I deleted it now, but in any case it was not active, so should not have been causing problems

Thanks for testing my configuration, but I guess mine was a more generic question: what is the best way to parse the debug log file and identify an error like mine? This started happening after I added a few domains to the whitelist, and it would be good to know how to find it if it happens again in the future

EDIT: I removed the weird config, restarted and

Service Status

Version 1.1.2-r20 - Active.
Blocking 187793 domains (with dnsmasq.servers).
Force DNS ports: 53 853.

Service Errors

Failed to process allow-list!
Errors encountered, please check the [README](https://docs.openwrt.melmac.net/adblock-fast/)!