Adblock support thread

¯\_(ツ)_/¯. It depends. For the most part, the basic dnsmasq package is fine.

Myself, I have the extra space and I have a couple of packages that require it so I’ve always used the dnsmasq-full package by default.

If you are running low on space and need it down the road your best bet would be to make a new build incorporating it rather than using opkg to remove dnsmasq and installing dnsmasq-full because you gain nothing on a squash fs.

Hi,
I just installed the package https-doh-proxy and now my adblock lists are not loading any more. (Loading never stops, 0 Sources loaded, no error messages from syslog)

Https-DoH-Proxy settings:

config main 'config'
	option canary_domains_icloud '1'
	option canary_domains_mozilla '1'
	option dnsmasq_config_update '*'
	option force_dns '1'
	list force_dns_port '53'
	list force_dns_port '853'
	option procd_trigger_wan6 '0'

config https-dns-proxy
	option resolver_url 'https://dns.digitale-gesellschaft.ch/dns-query'

config https-dns-proxy
	option resolver_url 'https://doh.ffmuc.net/dns-query'

config https-dns-proxy
	option resolver_url 'https://doh.mullvad.net/dns-query'

config https-dns-proxy
	option resolver_url 'https://dns.adguard.com/dns-query'

My Adblock settings are as following:

config adblock 'global'
	option adb_enabled '1'
	option adb_debug '1'
	option adb_forcedns '1'
	option adb_safesearch '0'
	option adb_dnsfilereset '0'
	option adb_mail '0'
	option adb_backup '1'
	option adb_maxqueue '4'
	option adb_trigger 'wan6'
	option adb_dns 'dnsmasq'
	option adb_fetchutil 'curl'
	option adb_repiface 'br-lan'
	list adb_stb_sources 'alternates/fakenews-gambling-social/hosts'
	option adb_report '1'
	option adb_backupdir '/etc/adblock_lists'
	list adb_sources 'adaway'
	list adb_sources 'adguard'
	list adb_sources 'adguard_tracking'
	list adb_sources 'android_tracking'
	list adb_sources 'anti_ad'
	list adb_sources 'anudeep'
	list adb_sources 'bitcoin'
	list adb_sources 'disconnect'
	list adb_sources 'games_tracking'
	list adb_sources 'openphish'
	list adb_sources 'phishing_army'
	list adb_sources 'smarttv_tracking'
	list adb_sources 'spam404'
	list adb_sources 'stevenblack'
	list adb_sources 'wally3k'
	list adb_sources 'whocares'
	list adb_sources 'winhelp'
	list adb_sources 'winspy'
	list adb_sources 'yoyo'
	option adb_dnsinstance '0'
	list adb_zonelist 'lan'
	list adb_portlist '853'
	list adb_portlist '53'

Anybody had this issues as well or knows a workaround?
Maybe good to mention: I recently installed ds-lite as well, after changing my isp, which seems to cause problems with many apps, I don't know if this is related.

Are you sure you are using https-doh-proxy and not https-dns-proxy? Even if you are using https-dns-proxy, your config looks very incomplete.

Check first if https-dns-proxy is working before checking the issue with adblock. I have the same setup and I haven't had issues with it. Below is what my https-dns-proxy config looks like:

config main 'config'
	option canary_domains_icloud '1'
	option canary_domains_mozilla '1'
	option dnsmasq_config_update '*'
	option force_dns '1'
	list force_dns_port '53'
	list force_dns_port '5353'
	list force_dns_port '853'
	option procd_trigger_wan6 '0'

config https-dns-proxy
	option bootstrap_dns '1.1.1.1,1.0.0.1'
	option resolver_url 'https://security.cloudflare-dns.com/dns-query'
	option listen_addr '127.0.0.1'
	option listen_port '5053'
	option user 'nobody'
	option group 'nogroup'

config https-dns-proxy
	option bootstrap_dns '8.8.8.8,8.8.4.4'
	option resolver_url 'https://dns.quad9.net/dns-query'
	option listen_addr '127.0.0.1'
	option listen_port '5054'
	option user 'nobody'
	option group 'nogroup'

And by the way, disable the adb_forcedns in AdBlock, as it is redundant. HTTPS DNS Proxy will handle that already.

1 Like

Hi @dibdot, can you help me out with the Adblock Reports (if you have time)?

After changing the parameters (from default) to the following:

	option adb_repchunkcnt '10'
	option adb_repchunksize '10'
	option adb_repiface 'any'

The DNS Report is just showing no values.

I've checked the PCAP files and it does have DNS queries from both of my lan networks (lan and guest).

Additional Details:
tcpdump - 4.99.4-1
adblcok - 4.1.5-8
kernel - 6.1.78 (using custom build snapshot)
arch/platform - ramips/mt7621
memory: 512mb (with 40-44% free)

Thanks in advance!

1 Like

Oh, yeah I mistyped, the package name in question is indeed https-dns-proxy.
And it works well when activated (I just disabled the default gateways). Most of the gateway settings you send are the defaults anyways and don't need an entry. I added some bootstrap_dns settings now as well.

My setup works, there are no dns leaks or anything. It's just that adblock gets stuck at loading and sometimes merging lists.

This is the last line I get from the log-view:

Mon Feb 26 04:32:00 2024 user.debug adblock-4.1.5[4461]: f_list   ::: name: yoyo, mode: restore, cnt: 3760, in_rc: 4, out_rc: 0
Mon Feb 26 04:32:05 2024 user.debug adblock-4.1.5[4461]: f_list   ::: name: -, mode: merge, cnt: 525826, in_rc: 4, out_rc: 0

It works well enough if I boot openwrt with the service https-dns-proxy disabled.

Maybe I should install a cronjob to disable https-dns-proxy on shutdown and enable it only 5 minutes after boot or so?

Hello there,

I am trying to setup Adblock with unbound (dnsmasq removed). My problem is that the backup list (from the previous session) does not work (or is not used) on a cold start -while- the new list is constructed. In the time the new list is being constructed - no blocking takes place at all; certainly not with the previous session's block list. My understanding is that the backup block list from the previous session is to be used while generating a new list -if- option adb_backup is set to '1'. If I have that wrong let me know.

My test was:

  • Reboot my router
  • Once the router issues a DHCP address and is accessible (the minimum is ready), I can ping a site that should be blocked.
  • Once the -new- list is ready the same ping fails as expected. This rebuild can take around 10 minutes or so. Occasionally the rebuild locks up for hours, or it finishes with no blocked sites.

I have fiddled with several options and scenarios. I see there are more options in /etc/config/adblock than are documented on:
https://github.com/openwrt/packages/blob/master/net/adblock/files/README.md

On another machine I have used the same basic approach on OpenBSD for a good year now (I know it is BSD vs Linux etc)
https://codeberg.org/unixsheikh/dnsblockbuster

For an half baked solution I am pondering using the same basic script form this codeberg.org page and just making a fixed blacklist. I know it will not update with each boot, but it can be updated manually once a month or so.

Naturally I have searched and looked over this forum (and a few others that are related). There are a few closely related cases, but not exactly this issue; adblock backup problem with unbound.

What have I overlooked?
Or does the use unbound make adblock awkward?

Thanks for time and effort folks.

George

Netgear Nighhawk R7200, AC2100, V1

Powered by LuCI openwrt-22.03 branch (git-24.048.61459-e4b384c)
OpenWrt 22.03.5 r20134-5f15225c1e 

System
Hostname           test
Model              Netgear R7200
Architecture       MediaTek MT7621 ver:1 eco:3
Target Platform    ramips/mt7621
Firmware Version   OpenWrt 22.03.5 r20134-5f15225c1e / LuCI openwrt-22.03 branch git-24.048.61459-e4b384c
Kernel Version     5.10.176
Local Time         2024-02-26 15:13:44


Adblock
Configuration of the adblock package to block ad/abuse domains by using DNS. For further information check the online documentation
Information
Status / Version     enabled / 4.1.5
Blocked Domains      141,514
Active Sources       easylist, notracking, stevenblack
DNS Backend          unbound (unbound-control), /var/lib/unbound
Run Utils            download: /bin/uclient-fetch, sort: /usr/libexec/sort-coreutils, awk: /bin/busybox
Run Interfaces       trigger: lan, report: br-lan
Run Directories      base: /tmp, backup: /tmp/adblock-Backup, report: /tmp/adblock-Report, jail: /tmp
Run Flags            backup: ✔, flush: ✘, force: ✔, search: ✘, report: ✔, mail: ✘, jail: ✘
Last Run             start, 12m 53s, 250/149/133, 2024-02-26T15:01:02-05:00


config adblock 'global'
    option adb_enabled '1'
    option adb_debug '0'
    option adb_safesearch '0'
    option adb_dnsfilereset '0'
    option adb_mail '0'
    option adb_backup '1'
    option adb_dns 'unbound'
    option adb_fetchutil 'uclient-fetch'
    list adb_sources 'easylist'
    list adb_sources 'notracking'
    list adb_sources 'stevenblack'
    list adb_stb_sources 'alternates/porn/hosts'
    list adb_stb_sources 'hosts'
    list adb_stb_sources 'alternates/gambling/hosts'
    list adb_stb_sources 'alternates/porn-social/hosts'
    list adb_stb_sources 'alternates/gambling-porn-social/hosts'
    option adb_forcedns '1'
    list adb_zonelist 'lan'
    list adb_portlist '53'
    list adb_portlist '853'
    list adb_portlist '5353'
    option adb_trigger 'lan'
    option adb_report '1'
    option adb_repiface 'br-lan'

Hi, just to confirm, your purpose is to have the adblock list running during a router start/restart? If this is the case, the block list generated file needs to survive the restart/reboot. This is usually done by pointing both the "DNS Directory" (where the adb_list.overall will reside) and the backup directory to a permanent storage location (a USB drive for example). For routers, it is not recommended it to store it in the internal storage as this will eat up the maximum write cycles on the flash memory overtime.

I see that your router has a USB, you can use a USB flash drive for this purpose. You need also to make sure that the USB drive is properly configured so it will be auto-mounted.

On my setup, I've mapped it to a directory to a mounted flash drive. Below is how it looks like in my setup.
image
image

option adb_backupdir '/tmp/usb/openwrt/adblock/backup'
option adb_dnsdir '/tmp/usb/openwrt/adblock/dhcp'

Did you use tcpdump-full or the tcpdump-mini variant?

By default the backup directory (adb_backupdir) points to /tmp (RAM) which is gone on reboot. Just point your backup directory to an usb stick.

Sorry forgot to mention.. i'm using tcpdump-full

Do you want me to send PCAP files?

Thanks, it's reproducible and fixed with adblock 4.1.5-9.

1 Like

Super thanks! Looking forward for the update. :grinning:

Thanks a lot. Can confirm update fixed the issue.

Thanks again!

1 Like

Thanks for the feedback folks.

I understand the reasoning that a block list be kept on a USB stick etc, but I see other files like /var/lib/unbound/unbound.conf are updated and rewritten to flash on every boot. The date and time is updated -in- the file at the top on every boot and it survives when power is removed, so they are not in a RAM-overlay but in flash.

Anyway; I removed luci-app-adblock and adblock and I am now (for the fun of it) tying to setup a simple unbound fixed block list (saved in flash) of my own generation. (I can manually update the block file once a month or so.) My problem is that the contents of /var/lib/unbound is, let me say, 'reset' on every boot even without adblock, so I assume it is in an overlay type of structure.

Is there a way in this OS to:

  1. See what is overlayed and what isn't (no sense editing something this is overlayed)

  2. IF something is overlayed, how do we get around the overlay temporarily to edit things, then restore the overlay (I know one can temporarily boot another OS to edit the first etc but it would be 'convenient' if it could be done from in the same OS) (I also don't want to recompile at this point, but just, e.g., flip configuration switches and edit if possible.)

If there is a relevant page/link on this forum that I have overlooked, just posting the URL would be adequate.

Oh - now that I am no longer using adblock, if you don't want this question on this page just say so and I can move it.

Thanks for your time and this neat project.

as said before, all this happen in RAM/tmpfs, e.g.:

root@blackhole:/var/lib$ df /var/lib/unbound/
Filesystem           1K-blocks      Used Available Use% Mounted on
tmpfs                  1020180     16820   1003360   2% /tmp

Oh it is happening in RAM, that is fine, but at -some- point (for one example) it appears that the file /var/lib/unbound/unbound.conf is rewritten to flash. Even after not being powered for a week (I'm talking about another router) then powered up (with no internet connection), the date when Adblock last ran successfully updating the block list, is recorded at the top of unbound.conf.

After a week without power a RAM overlay would not contain much any longer.

That is one example why I say it appears to me that at some point something is altering flash, and that is all I am getting at for now. The fine details of this is what I am digging into for now.

Now if I am overlooking some detail of the overlay implementation that explains this, just says so. For example (just an idea) if I ssh into this router, does cat/less/vi not show me the real content of the /var/lib/unbound/unbound.conf file as it is in flash (perhaps copied to RAM etc, there are different ways to do overlays)?

In other words, could the first two lines in /var/lib/unbound/unbound.conf be altered on their way to the RAM overlay before I can view it (to simplify the text for now)?

Thanks,

Hello

I would like test uci-app-adblock but I have a question: in addition to all the hosts here: https://github.com/openwrt/packages/blob/master/net/adblock/files/README.md can we add our own ?

thanks

Yes.. it's documented in the README.

Hello there!

I just flashed
OpenWrt 23.05.3 r23809-234f1a2efa / LuCI openwrt-23.05 branch git-24.073.29889-cd7e519
(latest stable release) on my Cudy WR3000(v1). Everthing worked fine.

Then I installed adblock 4.1.5-9 and luci-add-adblock git-24.086.45142-09d5a38.

Ad-Blocking works, but when I select "Log View" in Luci I get

NetworkError

HTTP error 404 while loading class file "/luci-static/resources/tools/views.js?v=git-24.073.29889-cd7e519" at compileClass (http://192.168.100.1/luci-static/resources/luci.js?v=git-24.073.29889-cd7e519:168:16)

Any idea what's happening? Doesn't happen on my x86 with OpenWRT 23.5.0 and adblock 4.1.5-8 / luci-app-adblock git-23.189.72983-3072876.

Thanks and greetings from Germany!

René

EDIT: Think I fixed it. As I suspected some problems in LUCI, I used opkg upgrade to update all installed LUCI-packages. Yes, I know one normally shouldn't use opkg upgrade.

Yep, some times ago the adblock logread implementation has been shifted/centralized to LuCI base (to mitigate duplicate code in different apps which are using a similar logreader ... now it was backported to 23.05 (see https://github.com/openwrt/luci/commit/09d5a38a1fd85334030613696273e0828b94796b#diff-356190d8fb2cb55773fd66ed297cc635eddca09777622daab44a16c978931f27 for reference). That means you have to update luci-base as well ...