I'm quite excited about this idea of an exclusive allowlist. Seems like a cool feature if we can make it work.
Maybe allowlist_only_mode then, for clarity.
Would you like to implement it?
Does that beat exclusive_allowlist or exclusive_allowlist_mode? Maybe it does.
Personally, I find allowlist_only_mode the easiest to understand but I'm not a native English speaker.
2 Likes
@Wizballs? Which is your preferred from all the options so far?
@antonk I'd implement it gladly but it'd take me some time to get round to it. If you're keen then please go ahead but no pressure to do so.
1 Like
Seems to work.
https://github.com/friendly-bits/adblock-lean/tree/add-allowlist_only_mode
I also reshuffled the config entries a bit and updated the README. And in this revision the local lists are checked for rogue elements.
(Note that because of the current final checks, the code adds google.com, microsoft.com, amazon.com to the allowlist now - not sure if this is the right thing to do. maybe condition the domain checks on allowlist_only_mode being disabled?)
2 Likes
Excuse me, to jump in, just out of curiosity, because of my abandoned project "Parental Control", which (of course) also included DNS-based domain blocking. BUT in a completely different approach as yours. Actually I set up ad blocking only, about 750k domains, on 128MB device, about 20MB still free, running a custom built, shrinked down image.
Besides RAM usage, response times to the DNS enquiries should be important. Actually I see about 0.06sec for a blocked demain, and 0.09 for a resolved domain. For details, pls refere below.
Would you mind, to publish similar measurements for adblock-lean ?
Details of my environment:
phpLiteAdmin, table bl_ads:: Showing rows 755100 - 755111, Total: 755112
top
Mem: 100776K used, 22016K free, 42564K shrd, 0K buff, 66652K cached
time nslookup zzzkd.com
Server: 127.0.0.1
Address: 127.0.0.1:53
** server can't find zzzkd.com: NXDOMAIN
** server can't find zzzkd.com: NXDOMAIN
Command exited with non-zero status 1
real 0m 0.06s
user 0m 0.00s
sys 0m 0.00s
time nslookup bild.de
Server: 127.0.0.1
Address: 127.0.0.1:53
Non-authoritative answer:
Name: bild.de
Address: 145.243.240.20
Name: bild.de
Address: 145.243.248.20
Non-authoritative answer:
real 0m 0.09s
user 0m 0.00s
sys 0m 0.00s
cat board.json
{
"model": {
"id": "zbtlink,zbt-we826-16m",
"name": "Zbtlink ZBT-WE826 (16M)"
},
cat openwrt_release
DISTRIB_ID='OpenWrt'
DISTRIB_RELEASE='23.05.5'
DISTRIB_REVISION='r24106-10cc5fcd00'
DISTRIB_TARGET='ramips/mt7620'
DISTRIB_ARCH='mipsel_24kc'
1 Like
The measured times will heavily depend on the hardware. So I don't think results will be directly comparable to yours. If you would like to compare on the same machine, you are most welcome to do so and let us know.
1 Like
This is very impressive, I wonder how you implemented this?
Edit: Ah, I suppose this explains it. Still impressive.
1 Like
Bruh, apparently I forgot to sync my fork to master before implementing add-allowlist_only_mode, and now there is a mess in commits. I'll recreate the branch and post a link soon.
1 Like
I think rendering checks conditional would be the right approach.
Or maybe this needs further thought. It never occurred to me that users might want to block or might not have access to those corporate sites. Hmm.
Should we render checks conditional and make these domains configurable?
Or actually how about just configurable since then user could simply delete them all so a list with zero entries or change to appropriate allowed domains?
1 Like
I used a 560MHz device. May be, you have something similar in speed. Also possible to normalize another clock speed. My approach includes a data base, about 45MB for the blocklist, plus 30MB for additional packages.
1 Like
Yep. These checks have their disadvantages, including:
- In some countries some of these domains may be blocked by the government
- Some people use all sorts of parental'ish controls with their Internet providers, which may be blocking any of these domains as well
- These domains are not guaranteed to always resolve correctly by all dns servers
This is perhaps the least of all evils but some fundamental issues still remain (in particular, what happens if the user configures some local domains and then they fail to resolve because of a temporary DNS problem?) I'm wondering whether these checks are actually needed at all?
Anyway, here is the re-created branch:
https://github.com/friendly-bits/adblock-lean/tree/add-allowlist_only_mode
CPU speed is not expressed in MHz. There is also instruction set, cache size and all sorts of architectural differences. Personally, I'm not interested in making these measurements to compare to your results. If you are interested then it's on you to measure and compare, and again - it makes much more sense to compare on the same hardware.
2 Likes
My results with- and without adblock-lean
Checking active blocklist.
Active blocklist check passed with the new blocklist file.
New blocklist installed with entries count: 717,758.
Results with adblock-lean started
$ time nslookup zzzkd.com
Server: 127.0.0.53
Address: 127.0.0.53#53
Non-authoritative answer:
Name: zzzkd.com
Address: 64.225.25.106
** server can't find zzzkd.com: NXDOMAIN
real 0m0.024s
user 0m0.010s
sys 0m0.011s
$ time nslookup bild.de
Server: 127.0.0.53
Address: 127.0.0.53#53
Non-authoritative answer:
Name: bild.de
Address: 145.243.248.20
Name: bild.de
Address: 145.243.240.20
real 0m0.023s
user 0m0.012s
sys 0m0.008s
Results with adblock-lean stopped
$ time nslookup zzzkd.com
Server: 127.0.0.53
Address: 127.0.0.53#53
Non-authoritative answer:
Name: zzzkd.com
Address: 64.225.25.106
real 0m0.049s
user 0m0.014s
sys 0m0.011s
$ time nslookup bild.de
Server: 127.0.0.53
Address: 127.0.0.53#53
Non-authoritative answer:
Name: bild.de
Address: 145.243.248.20
Name: bild.de
Address: 145.243.240.20
real 0m0.023s
user 0m0.009s
sys 0m0.012s
3 Likes
I personally think it's good to check that domain lookups to certain domains still work to safeguard against, for example, an over-zealous blocklist. Bear in mind that these are updated daily and mistakes can happen. We've seen issues with OISD crop up now and then.
Albeit I recognise that it may be slightly awkward to have user have to delete the domain check entries in the config if the allowlist only mode is used.
If we keep the domain checks then it seems to make sense to make the list of domains to check configurable.
It seems that our options are:
- render domain checks conditional on allowlist only mode being deactivated;
- rely on configurability of the list of domains to check such that user can delete all entries to disable the check; or
- abandon domain checks altogether.
And unless we abandon the domain checks altogether, it seems making the domains configurable makes sense anyway.
1 Like
All right, I suppose let's add an option to configure test domains. We could automatically add the test domains to the allowlist just like the current revision does, so the user wouldn't need to delete them from the config.
Another thing: when allowlist_only_mode is enabled, the current revision ignores the local blocklist and any configured blocklist URLs except ipv4 blocklist. My logic was that if everything besides the allowlist domains is blocked anyway then there is no reason to waste resources on processing the blocklist parts. However thinking about it again, I'm realizing that the user may want to selectively block certain subdomains of allowed domains, so perhaps blocklist parts shouldn't be ignored, after all. However, the way adblock-lean is currently working, I believe that allow will take precedence under these circumstances. So the subdomains included in the blocklist won't be blocked. So perhaps to implement this properly, we should reverse the precedence in this mode. Maybe simply bypassing the awk command which does allowlist domains removal from the blocklist will do it?
2 Likes
This is the one! Easy to understand
3 Likes
What if user wants to block google.com? Then he'd just have to alter the configured list of domains to check I suppose. So that scenario is still OK.
2 Likes