Adblock support thread

Just download the ipk package files (referenced in the first post) directly to your router and install them via opkg.

1 Like

Thanks I was able to update it.

Noticed config file already had adb_report which I changed from 0 to 1 but still the tab didn't come up.

tcpdump has been installed as well? If so, please reset your luci cache (rm -rf /tmp/luci-*)

1 Like

Unfortunately this did not help. I do have tcpdump (version 4.9.2-1) installed and did luci cache reset too.

Below is my config

config adblock 'global'
	option adb_dns 'dnsmasq'
	option adb_fetchutil 'uclient-fetch'
	option adb_trigger 'wan'
	option adb_enabled '1'

config adblock 'extra'
	option adb_report '1'
	option adb_maxqueue '8'
	option adb_debug '1'
	option adb_nice '0'
	option adb_forcedns '1'
	option adb_forcesrt '0'
	option adb_dnsflush '1'
Adblock Version 3.6.5-2
luci-app-adblock git-19.190.55614-35357e4-1

You haven't updated your luci adblock app - you're stil on 18.06.x ... please use the latest snapshot version as well (see first post).

1 Like

Perfect that worked. Thanks!

I have a question regarding the performance impact on DNS queries when running Adblock. While my router with 128MB RAM can easily handle relatively large adblock lists, I am noticing a lineair performance loss depending on list size, or a O(n) complexity. I am using dnsmasq as the DNS forwarder.

Is there a way to make dnsmasq use a hashtable for looking up entries in the list, and hence a O(1) complexity, or put differently, DNS performance independent of list size? If not, is it worth switching to the unbound backend? Ie does unbound support hashtables for the lookup?

1 Like

Everything just feels faster with unbound.

Do you use trunk builds?
Just of curiosity does adding:
option min_cache_ttl '1800'
to /etc/config/dhcp under config dnsmasq make things a little bit better?
For example, some sites like google use very small TTL values of like 5min.

Adding:
option nonegcache '0'
option cachesize '10000'
Maybe is also worth a try.

Changing caching behavior might mask the problem in some cases, but it will never improve uncached results, which will still suffer a pretty severe performance penalty due to the slow O(n) lookups. I really want my uncached results to be O(1) through proper use of hashtables.

Does it feel faster, or do you also happen to have objective performance figures?

1 Like

No, I have not benchmarked it :wink:
The first lookup with unbound can be slow too.
But after the cache is build up everything loads instantly.
You can make a script to save and load the cache. (using unbound-control)
When you have to reboot the device you can load cache.
I also have prefetching enabled and set a min cache TTL of 1h.
But unbound also uses a little bit more of ram.

1 Like

I have found an interesting topic here: https://www.reddit.com/r/sysadmin/comments/beqbcj/dnsmasq_is_very_slow_due_to_adblocking/

Apparently, using dnsmasq's addn-hosts is much faster than address. Do you happen what is used in Adblock's case?

1 Like

I think adblock uses serversfile (--servers-file) approach.

How would that compare to addn-hosts?

Edit: To give you an idea about the kind of results I am seeing. First adblock enabled with 140k domains:

Screenshot%20from%202019-08-06%2018-41-17

Adblock disabled:

Screenshot%20from%202019-08-06%2018-43-11

1 Like

http://www.thekelleys.org.uk/dnsmasq/docs/dnsmasq-man.html

--addn-hosts

  • Maps hostnames/domains to IPs. (Queries are never not forwarded to upstream resolver)
  • Specifying no IP results in NXDOMAIN
  • Needs two entries per host/domain (IPv4/IPv6)

--server

  • Allows to assign/map domains to a specific upstream resolver.
  • Specifying no upstream resolver results in NXDOMAIN
  • Only one entry per host/domain

Which one is faster? I don't know :thinking:

1 Like

I have exactly the same: adblock is extremely slow. There is room for improvement, for example using external Pi-Hole is much faster.

I put it to the test. I fetched the list adblock created from the router. I then edited into the host file format, and uploaded it back unto the router. I then manually imported the host file in dnsmasq to make sure addn-hosts is used. Lo and behold, adblocking was working, and DNS performance was not impacted at all!

@dibdot would it be possible to change the workings of this wonderful adblocking package to create and import a list via addn-hosts instead of the currently used method? The speed-up is insane. It now performs the same as pi-hole with very large lists!

@shm0 Where did you find above documentation? I am unable to create a file by specifying no IP (which should result in NXDOMAIN). This forces me to add each domain twice to the list, once for IPv4 and once for IPv6. While it works, the list is needlessly larger, which is a bummer with the constrained space routers have.

For my tests I had to use a IPv4 only list, or the list wouldn't fit on the router. To test whether it was blocked properly I used ping -4 to force IPv4 pings to a few domains on the list. They all resolved to 127.0.0.1 instead of NXDOMAIN, which isn't ideal. To test the performance, I reran the benchmark, which showed unimpaired performance compared to not running the host file.

Host file that I used (140k entries): https://we.tl/t-OTDqjdx0aS

2 Likes

In principle it's no problem to add another dnsmasq output variant. Said that, did the described approach deliver "NXDOMAIN" in return? What about IPv4/IPv6? If I remember correctly, the addn-host-method blocks only on single host level, e.g. 'doubleclick.net' blocks only this host and not 'www.doubleclick.net', is this assumption correct? If so, then this method probably leads into much bigger lists with serious memory constraints ...

No, unfortunately via the manually crafted host file as posted above, localhost is returned. Having said that, according to the following documentation posted by @shm0 it should be possible. I just haven't managed to find the correct format/syntax of the host file:

Unfortunately, both IPv4 and IPv6 require a separate entry (=big host file) according to the following documentation:

Having said that, seeing as specifying no IP should result in NXDOMAIN, I am hoping this would work for both at the same time (it would only make sense). I am still struggling with getting the format/syntax correct though.

That is correct unfortunately.

  1. Is there even a list available that contains all subdomains?
  2. How much larger would you expect the list to be?

I have a few questions for people following this discussion:

  1. Does anyone happen to know the correct syntax for addn-hosts to return NXDOMAIN?
  2. Does anyone happen to know how the filtering is done for Pi-hole? They used to use dnsmasq, and are now using a dnsmasq fork. The Raspberry Pi 1st generation is really slow, yet more than adequate for large lists. Does anyone know how they handle the blocking? At first sight, addn-hosts seems to have quite a few large drawbacks, but then again, so does having multi-second DNS resolution times as can be seen from my benchmarks above.
1 Like

http://www.thekelleys.org.uk/dnsmasq/docs/dnsmasq-man.html

-A, --address=/[/...]/[]

...
To include multiple IP addresses for a single query, use --addn-hosts= instead.
...
As for --server , one or more domains with no address returns a no-such-domain answer, so --address=/example.com/ is equivalent to --server=/example.com/ and returns NXDOMAIN for example.com and all its subdomains. An address specified as '#' translates to the NULL address of 0.0.0.0 and its IPv6 equivalent of :: so --address=/example.com/# will return NULL addresses for example.com and its subdomains.

Did you try to create an addn-hosts file in the following format?
address=/example.com/
address=/example2.com/
address=/example3.com/
.
.
.

I am fairly sure I did, because that's the format adblock uses by default (or am I mixing things up?). I tried to load that one directly via addn-hosts, and it would fail.