Adblock-fast: ad-blocking service for dnsmasq, smartdns and unbound

Hello @dave14305 I took a shot at supporting the allow-listing by explicitly adding the domains to the final dnsmasq.servers file like you suggested in 1.1.2-6 available from dev repo.

It of course took quite a bit longer to implement as I originally hoped, I needed to update both processing of allow/block-lists when downloading, the operation of allow command and even make changes into how the size of the final block-list is calculated, as it would now contain explicitly allowed domains.

Please test 1.1.2-6 as much as you can. I still need to probably add support for allowing domains when using remote dnsmasq config file, but I'd like to have this basic functionality tested.

Great, I’ll be able to test it out tomorrow evening when I have the network to myself.

I was able to install and start some testing during Monday Night Football.

At first glance, I’m surprised to see that the allowing is only one method or the other (e.g. sed or #), but I was advocating to do both methods at once.

My first test using Hagezi Pro was to allow apple.com. This didn’t allow any of the currently blocked Apple sub-domains because the blocked domains are more specific than the new allow entry. You would need to sed out all blocked entries ending in apple.com and then add the new entry server=/apple.com/#. The sed is technically enough for this particular scenario (allowing a higher-level domain of one or more blocked domains), but since you won’t know in advance if it’s a partial match, exact match or no match, you should execute both methods.

The next test was to whitelist a random doubleclick sub-domain 9120651.fls.doubleclick.net since fls.doubleclick.net is in the block list. This works as expected. 9120651 resolves properly, and any other subdomain like 9121695.fls.doubleclick.net continues to be blocked.

Allowing an exact match of blocked domain like browser-intake-datadoghq.com works as expected.

Does all that make sense?

@dave14305 I thought it would be a significant speed improvement if I didn't need to purge the final block-list from allowed sites, but I see your point. So 1.1.2-7 available from OpenWrt snapshots repo and from my private repo has the proper processing as you suggested.

Something is wrong since the sed patterns are at the top of the file.

root@router:~# head /var/run/adblock-fast/dnsmasq.servers 
/(^|\.)cdn\.jsdelivr\.net$/d;
/(^|\.)browser-intake-datadoghq\.com$/d;
server=/cdn.jsdelivr.net/#
server=/browser-intake-datadoghq.com/#

I imagine this is the relevant line:

I see now, $SED_TMP is appended with the server statements, but it still has content in it from earlier.

1 Like

This is embarrassing. Fixed in adblock-fast 1.1.2-8.

2 Likes

On the plus side, everything is working well based on my 3 alllowlist scenarios! Thank you.

I have previously ignored the news about the per-instance conf-dir support for dnsmasq (applicable for dnsmasq.conf, dnsmasq.ipset and dnsmasq.nftset configurations only), but was reminded about it with the recent drama with switching default values for per-instance conf-dir.

The adblock-fast 1.1.2-10 (and please update the luci app to the same version), has support with configurable per-instance conf-dir for dnsmasq. I've quickly tested with all 3 options affected (and dnsmasq.servers which is not affected) and it seemed to work for me.

If you use adblock-fast with either dnsmasq.conf, dnsmasq.ipset or dnsmasq.nftset options for resolver, especially if you have multiple dnsmasq instances, please install 1.1.2-10 and test.

Pending any reported bugs, I'll be creating PRs for OpenWrt packages both snapshots and 23.05 repos before the end of the week.

1 Like

I observed some “Unknown warning” lines in LuCI after installing a new build from sources. I neglected to install the recommended packages (gawk, grep, sed, coreutils-sort) and get the warning as expected.


Service Warnings        Some recommended packages are missing
                        Unknown warning
                        Unknown warning
                        Unknown warning

I thought the extra warnings odd and checked the json response:

"warnings": [
        {
                "id": "warningMissingRecommendedPackages",
                "extra": "gawk"
        },
        {
                "id": "grep",
                "extra": ""
        },
        {
                "id": "sed",
                "extra": ""
        },
        {
                "id": "coreutils-sort",
                "extra": ""
        }
],

The additional packages each create a separate id, and so the id doesn’t match a pre-defined warning text. Not a showstopper, but perhaps the json needs tweaking if multiple packages are missing?

This is a build off snapshot (actually the pesa1234 build for MT6000, but same official “packages” repo for Adblock-fast).

1 Like

Thanks @dave14305, was super-easy to fix since you've done all the work diagnosing it!

Fixed in adblock-fast 1.1.2-12, due to the modified i8n strings, would also require update of the luci-app.

The APK-compatible binaries can be found at https://dev.melmac.net/apk/

This is how the fixed luci app looks:

1 Like

Sorry, I’m in no-man’s-land between opkg and apk right now. Have you pushed the source anywhere yet?

Nevermind, remembered the old repo path for ipk.

apk repo/binaries: https://dev.melmac.net/apk/
opkg/ipk repo/binaries: https://dev.melmac.net/repo/

2 Likes

Running a x86 master image, issuing

/etc/init.d/adblock-fast stop

deletes the config file; apk is a bit borked so version is whatever is currently in master.

1 Like

Did you happen to be in the config directory when you ran the stop?

I wanted to test this, so I copied my config file to ~/adblock-fast and then ran the stop and afterwards, the config file was still in /etc/config, but the backup in my current directory had disappeared.

Yep, current dir was /etc/config.

@stangri the dnsmasqConfFile without a path causes this here:

+ '[' dnsmasq.servers '=' dnsmasq.conf ]
+ rm -f adblock-fast /var/run/adblock-fast/dnsmasq.conf.cache /etc/adblock-fast.dnsmasq.conf.gz
1 Like

Thanks @anomeome for posting and @dave14305 for investigating, it did sound like a bigger issue than just apk compatibility and I'll take @dave14305 findings to fix this, hopefully over the weekend!

1 Like

Looks like StevenBlack's hosts file currently has some erroneous entries which break dns resolution if used.

Disable that source (if you're using it) and update to adblock-fast 1.1.2-16 for fix.

I'm currently trying to setup adblock-fast on a second dnsmasq instance that is only used by specific devices.
I also have stubby running which uses 127.0.01#5453 (+ipv6) on the main instance.

I don't get what im doing wrong, I also don't get how to configure the ports correctly. The second dnsmasq instance (where adblock-fast is pointing) needs a different dns port because you can't run multiple thongs on port 53. What port do I pick? How is this related to the force dns port option? Do I need to setup some firewall rules?

Custom dns on the LAN interface is clear, custom dns on WAN points to cloudflare.

Would love some help.

(dnsmasq-full is installed)

The way hijacking is implemented in adblock-fast, it maps the hijacked port from the requests to an open port on the router or drops the requests if this port is not open on the router. In your case, the clients are likely to send requests to hard-coded DNS servers still on port 53, whereas the target dnsmasq instance is not going to be running on port 53, so yes, you may have to set up the hijack firewall rules manually. Maybe it would be easier to configure adblock-fast on the primary (port 53) dnsmasq instance and set your LAN clients where you don't need adblocking to use an alternative instance.