Adblock-lean: set up adblock using dnsmasq blocklist

[antonk]

@seishinui

a manual scheduled task to run the update -y

On one hand, I'm happy that we didn't implement this option for nothing (i.e. that someone is actually using it), on the other hand - updating anything on a complete autopilot is risky, especially so updating an app which manages a critical system service (i.e. DNS resolver on a router). Also keep in mind that adblock-lean is a small indie project, maintained by a couple of guys (myself and @Lynx). While we are doing our best to test before making a new release, we are just two people and at worst no one else, at best a couple volunteers help us with testing. And while we have invested (and continuing to invest) a lot of effort in error checking and handling, fallback mechanisms etc, there is no way for us to guarantee that an update will not break anything.

@seishinui

add an enable tag/option for it first, then another check to see if the dir is set otherwise enable is ignored

Yeah, this is another thing I was thinking about - that the user should explicitly enable this functionality, not just provide a path. So in the meantime, I replaced the option PERM_BLOCKLIST_IS_MAIN with the option PERM_BLOCKLIST_MODE. This is the description which I'm planning to include for this option:

# Governs whether and how permanent blocklist is used
# 'disable' (default): permanent blocklist will not be used. The blocklist file will be stored on the ramdisk.
# 'manual': directory specified in PERM_BLOCKLIST_DIR will be checked for file named 'abl-blocklist' (with or without extension '.gz' or '.zst') -
#   if found, that blocklist will be loaded at boot (rather than downloading, processing and loading a new blocklist)
#   but adblock-lean will not create or update that file (useful to prevent flash wear, e.g. when the permanent blocklist is stored on the built-in flash of a router).
#   If not found, adblock-lean will act as if mode is 'disable'.
# 'main': adblock-lean will use the directory specified in PERM_BLOCKLIST_DIR to store and update the blocklist file
#   and no additional blocklist will be stored on the ramdisk.
#   If the directory is inaccessible, adblock-lean will fall back to using the ramdisk.

(this is a copy-paste from the comment which I'm planning to attach to the option in the config file)

Of course, I implemented handling of invalid/fault conditions, including when PERM_BLOCKLIST_MODE is set to manual or main but PERM_BLOCKLIST_DIR is not set or does not exist, as well as required addnmount entries checking and creation etc. There is a lot of logic to handle, but hopefully everything is covered at this point.

@seishinui

skip the initial startup blocklist update, so a blocklist update would only then happen at the cronjob schedule

Again, I agree - this is the best idea I could come up with as well.

@Lynx

I reckon it would be by establishing concurrent instance (supported in recent OpenWrt versions) and flipping between dnsmasq instances for incoming DNS requests.

This is the exact thing I don't think is possible. Putting aside the question of how to re-define dnsmasq instances on-the-fly (as those are normally defined in /etc/config/dhcp which is typically stored on the flash and rewriting that file frequently would accelerate flash wear), the immediate show-stopper is that a second instance of dnsmasq can not listen on the same port and on the same network interface. If you try to make it (I just tried), it will print this error:

dnsmasq: failed to create listening socket for 192.168.1.101: Address in use

and exit.

So I simply do not see a way to start loading the blocklist into instance X, start instance Y in parallel, and kill that instance when instance X is done loading the blocklist. Either X or Y will die with an error at some point.

[antonk]

@Lynx

Makes me wonder whether nftables could be leveraged in ways we haven't thought before. For example, seems one could use the marks to enable a two-tier blocklist (e.g. parent or child devices). I wonder even whether we could alter marks based on content of request using nftables.

Honestly, this is too complicated for my simple brain

That said, I've been thinking about implementing support for using actually different blocklists for different dnsmasq instances (thus enabling similar functionality). While this is also a complex thing to do, at least I can see a way forward with it.

[reinerotto]

Being a fan of KISS, I start my private adblock on very first boot (after flash) using a default, minimal blocklist, included in custom image ( or package).
On first blocklist update, this permanently stored blocklist is replaced.

Thus, only in the time period between boot and first blocklist update afterwards, system is using a somewhat "out of date" blocklist.
Assuming rather infrequent boots, this should not be an issue for long term ops.

[seishinui]

Goes to show without saying that I, and many, greatly appreciate the work and thank you for it even if unspoken at times. Seriously, thank you for the work.

@antonk

updating anything on a complete autopilot is risky

It is indeed risky in that sense, but for me, that time is when I’ll be physically available in the place as that’s when I do visit. I also regularly visit the forums so I know if some thing is off or needs to be done. It’s just a convenient thing for me that unless there’s an issue, then I won’t even have to spend time to ssh to the router nor even think about it when I visit.

@antonk

This is the description which I'm planning to include for this option

This is good and very understandable. I can only imagine how back and forth the work on the code is just to a handle a single logic more so if it’s a cascading logic, so again, greatly appreciate the work you do on that front.

[spence]

@antonk

Hi all, I'd like to discuss a new feature which has been asked for several times - the option to have the blocklist stored permanently and possibly that the permanently stored copy would be loaded at boot.

Hi,

I am strongly in favor of this.

Sorry that I have not taken time to digest it all and provide better feedback. Here are a couple thoughts. Some is a repeat of your proposal but omissions of other parts are not meant as negative feedback of other parts of your proposal.

Have the option to permanently store a blocklist and use it on boot.
Have the option to automatically update the permanent blocklist and define the replacement schedule, with the ability to set a specific time of day and which days of week or month etc. (easy with cron job?) Set up this logic to be simple and aimed at the existing crontab capabilities. Keep thundering herd protections or optionally toggle that off.

Have basic checks that a replacement list is sane to use, like number of list items before writing to storage. This is probably exactly as you already do. Keep previous list on any update failure. Be able to log success or failure and maybe list count etc.

I think enabling custom times for the update may be a suitable solution for the service outage for a lot of home use cases if for example set to human sleep times and avoiding relevant home-lab needs.

I think each ideas should be weighed in its effect on simplicity, efficiency, speed, feature, security etc keeping in line with the various aspects of “lean”.

Let me know if you want clarification on any of my thoughts on this new feature.

Last but NOT least, Thank You to maintainers and contributors. It has made this part of OpenWrt better for many.

[spence]

Another thought…

Be able to choose between:

• Daily Fetching and using an updated list in RAM as is current practice with the addition of a updated permanent list as boot/fallback, updated on a user defined schedule. The permanent list could come from the currently loaded list. Don’t fetch a new remote list on boot / power cycle / dnsmasq restart.
• Only load/use the permanent list with its update schedule.
• Don’t use a permanent list.
• Option to fail-over to option 2 if option 1 is set and fails.
• Option to fail-over to live fetching of remote list if option 2 is set and fails.

With ( so far ) reliable USB storage, I see myself using option 2 with a daily update of the permanent list on USB storage but would like fail-over to fetching the remote list on boot / restart if loading the permanent list fails. I’m not sure daily updates are even needed for my simple use case / threat model and might consider biweekly updates or something like that.

Maybe with modular code, functions etc, I’m hoping much of the fallback can be done with calls to routines for the other option, maybe with adding a minimal amount of parameters and logic.

This all just some ideas and not a feature request.

[antonk]

Thank you for your suggestions. I'm looking into implementing support for separate update schedule of the permanent blocklist but not sure this option will make it into the next release.