Hey, skipping the whole file containing rogue element has always been the approach. Adblock-lean never excluded just the single line. The logic is that if a blocklist contains a rogue entry, then assume that whole file is potentially a problem.
But do you have an example of what is happening for you?
Indeed - we never just removed a rogue element. I think the thinking was that rogue elements should really not crop up by definition and if they did one should be investigating!
Always open for discussion. If there are good reasons to offer the option to simply excise them we can easily implement this. But at least in my mind we shouldn’t just paper over such rogue elements because rogue elements can include nefarious tricks like redirecting a domain to an IP. So we should treat these seriously.
@HSK Right, I don't know how widely supported it's currently, which is why I asked this to the other person suggesting DNS over HTTPS. My comment about lack of support dates back to 2017/2018, on my old router. I didn't have any major reasons to re-consider encrypted name resolutions until now.
@Lynx It's more of a "time" thing. Time to setup and maintain yet another service. I'd hoped it would also provide DNS over TLS on its own, so I would need only one service for both features (adblocking and DNS over TLS).
I am already late with the release of my Wireless ARP Monitor application, so free time is really in short demand for me.
There isn't going to be any significant performance difference between DNS over HTTPS and DNS over TLS (DNS over TLS has less layers, and in theory should be faster).
DNSCrypt through dnscrypt-proxy2 might be slightly faster aswell as provide solutions to some of the shortcomings of the other two protocols. Less DNS servers support it though.
I can pretty much guarantee you that adblock-lean will never include a DoT inbuilt option. It's simply not in the scope of this project. And there is little need for it, given that stubby exists.
Everyone is short on time really, but the beauty of both adblock-lean and stubby is that once setup, neither require ongoing maintenance.
Always a question of how we prioritise things.
I setup stubby ages ago and it has just worked since. And adblock-lean surely only takes about 5-10 minutes to try (download, launch, tweak config potentially, launch again, uninstall).
I guess I drew conclusions too quickly. I had noticed my final blocklist had shrunk significantly recently, probably because complete blocklists are being ignored because of new rogue elements. Here are some examples:
Warning: Rogue element: 'server=/25z5g623wpqpdwis.onion.to/' identified originating in blocklist file part from: https://blocklistproject.github.io/Lists/dnsmasq-version/ransomware-dnsmasq.txt.
Warning: Rogue element: 'server=/10tradeoption.com/' identified originating in blocklist file part from: https://blocklistproject.github.io/Lists/dnsmasq-version/scam-dnsmasq.txt.
Warning: Rogue element: 'local=/pokolorujfb*.glt.pl/' identified originating in blocklist file part from: https://raw.githubusercontent.com/FiltersHeroes/KADhosts/master/KADdnsmasq.txt.
My reasons for thinking about skipping only the element is that, as in those cases, they are either improperly formatted or wrongly detected by adblock-lean, but they are clearly not malicious. I also see clearly the security issue with only skipping elements. I'm not sure what the best approach is while keeping things secure!
The first 2 are using outdated dnsmasq format, where instead of the keyword 'local' they are using the keyword 'server'. Currently our filter doesn't support this outdated format and hence these entries are flagged as rogue. In the upcoming release, we will transition to using raw domain lists rather than dnsmasq lists, so this sort of thing won't be an issue anymore, as long as you find an equivalent substitute for that blocklist in the raw domains format (these are sometimes also called "wildcard domains", of which there are 2 types, and the one which we will recommend to use with adblock-lean is the one which strangely has no or very few wildcards. it's a bit of a mess, I know - but the processing is faster for equivalent protection). If a substitute doesn't exist, you will be able to continue using the dnsmasq-formatted lists. We will consider allowing the above outdated format with them.
The 3rd one is flagged as rogue because it has the wildcard - * not in the very beginning of the URL. Generally wildcard processing is quite heavy on the CPU, plus they introduce some security issues, so we decided to only allow them only once in a domain name, and only in the beginning of it. We may reopen the discussion about that if this is important to you.
That said, did you consider using the Hagezi and/or the OISD lists?
Actually not necessarily outdated, they are still valid entries and can be used by dnsmasq. Earlier versions of adblock-lean would convert both "server" and "domain" (can't remember exactly, I think it was domain), to "local". It would probably be best to re-add this conversion to sed dnsmasq format lists.
This is technically a rogue entry even if we were allowing it before. From the dnsmasq instructions, dnsmasq only allows wildcards such as:
*.google.com
*.amazon.com.
etc
"Server" and "address" should be converted to "local"
Not domain as I incorrectly mentioned earlier
You guys should be proud to know that among adblock, adblock-fast, and adblock-lean, you are the only ones who fulfill all the allowlist scenarios I harped on before:
server=/my.ads.example.com/# when ads.example.com is blocked)server=/ads.example.com/#)example.com when ads.example.com and tracking.example.com are in the blocklist)Adblock only permits #1 and #2. Adblock-fast only permits #2 and #3. Adblock-lean permits #1, #2 and #3.
That's nice feedback cheers.
Last time I checked adblock-lean was the only one that could handle large allowlists also (eg 1,500 entries) extremely quickly.
Actually surprised the other two haven't "borrowed" this code...
Adblock does no line removal from the blocklist at all (only local=/.../#) and Adblock-fast compiles a sed script to remove allowed domains from the blocklist, but does no local=/.../#.
I think both are constrained somewhat by the desire to be able to serve multiple DNS backends.
I've had a look at this code before, it is very slow on large lists (1,500 allowlist entries, 900k blocklist). I terminated the script after 30 mins as it was still running. Approx 10 seconds on ablock-lean. We put of heeeeap of effort into getting allowlist's right...
They are different scope packages in that regard, and full respect for that type of flexibility.
Thanks for the clarifications. It is not especially important to me to allow more than one wildcard or to allow wildcards at the end, although I do enjoy using those blocklists. I do not think compromising on security for general usage is a good idea, but if the option was there to allow it in the config, I would probably use it.
As for the "server=" entries, I think it would be a good idea to re-add the conversion, unless it causes problems.
This shouldn't be a problem to get re implemented, let you know when.
I'm going to look into this a bit more. eg why this blocklist contains these entries etc. More curious than anything.