Adblock support thread

dibdot · 2019-08-16T04:32:34.788Z

Hi,

I've just pushed a new adblock release 3.8.0 to master (full change log see first post). If you would like to try the new version, you have to update your adblock config as well. Please use the --force-maintainer opkg option for that. After the LuCI package update please reset you LuCI cache with rm -rf /tmp/luci-*

Two major changes are:

add support for 'DNS File Reset', where the final DNS blockfile will be purged after DNS backend loading (save storage space). A small background service will be started to trace/handle dns backend reloads/restarts

In LuCI you'll find this new extra option:

luci-2.png890×105 6.91 KB

add support for the 'null' blocking variant in dnsmasq (via addn-hosts), which may provide better response times in dnsmasq

In LuCI you'll find this additional select box under the dns backend selection:

luci-1.png952×272 23.2 KB

Have fun!
Dirk

shm0 · 2019-08-16T19:37:40.421Z

Thanks for the update!
But why was the static blacklist removed from adblock conf?

dibdot · 2019-08-16T19:47:09.112Z

The blacklist is now treated like the whitelist ... always active, there were too many users who unintentionally deactivated this list to wonder afterwards, that entries on the blacklist don't work.

shm0 · 2019-08-16T21:28:59.204Z

Thank you for clearing that up!

ldir · 2019-08-17T11:37:41.276Z

I was confused why I couldn't get the DNS reporting to work. Looking in the script it turns out the 'reporting interface' is hard coded to 'br-lan'. Is there better way of changing this to 'eth1' other than hard coding the script?

My APU2 config doesn't use bridge interfaces for lan.

dibdot · 2019-08-17T14:20:24.934Z

Nope, 'br-lan' is just the default ... simply set 'adb_repiface' accordingly ...

repiface.png931×132 7.7 KB

ldir · 2019-08-17T21:16:26.207Z

ahh! additional parameters. ooops - thanks

RainGater · 2019-08-19T12:03:17.110Z

Updated to 3.8.0 and looks much better than 3.6.5-2 as it seems to update/refresh the lists faster. Thanks a ton.

Btw, I picked all the available blacklist sources (overall domain count: 185723) and my router is Netgear R7800. Would it cause any performance hit because I chose all the sources? I am not seeing any hit now as the available memory is still 333 MB/466 MB. Not bad and load is not severe yet. I picked both your tweaks: DNS File Reset & 'null' blocking variant in dnsmasq.

dibdot · 2019-08-19T13:47:01.900Z

Thanks for testing & your feedback!

RainGater:

Would it cause any performance hit because I chose all the sources?

I don't know. In my productive environments I'm using dnsmasq/null and unbound/nxdomain with round 50K domains and the new (still somewhat experimental "DNS File Reset") and see no downgrade performance wise.

WildByDesign · 2019-08-19T14:46:59.925Z

dibdot:

In my productive environments I'm using dnsmasq/null

Dirk, would you please be able to explain some more detail regarding this dnsmasq/null (I am assuming referring to addnhosts method) method?

I am not familiar with it, although I have read some of the recent discussions on how it may be faster as far as performance goes. However, I would really like to understand it better before I mess around with my config. I always have two instances of dnsmasq running with your adblock program and I don't want to do anything that would mess up my current setup.

Have you noticed better performance with dnsmasq/null?
How does the error messages in the browser differ between nxdomain and dnsmasq/null?

In my etc/config/dhcp file I have always had to manually add the following line:
option serversfile '/tmp/adb_list.overall'

That was specifically regarding my second dnsmasq instance only, of course.

If I were to switch to dnsmasq/null, for my second dnsmasq instance in etc/config/dhcp file, would I manually need to add a line for addnhosts method?

Thank you for your time, as always. It has been a pleasure testing your program over the past 3 years. I was actually looking through my archive of binaries and packages on my local storage and realized that I have stored every single build since adblock_1.0.0-1_all.ipk, including all of the various adblock2 testing files adblock_1.9.99-pre1-1_all.ipk all the way until these most recent build. Including, of course, all of the luci-app-adblock packages. It is crazy how time goes by so fast.

dibdot · 2019-08-20T08:33:39.888Z

WildByDesign:

Dirk, would you please be able to explain some more detail regarding this dnsmasq/null (I am assuming referring to addnhosts method) method?

The new blocking variant uses an unspecified IP address for blocking ("::" in IPv6 and "0.0.0.0" in IPv4). It must never be assigned to any node and indicates the absence of an address. If you dig for such address, you see this:

; <<>> DiG 9.11.5-P4-5.1+b1-Debian <<>> doubleclick.net
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 134
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;doubleclick.net.		IN	A

;; ANSWER SECTION:
doubleclick.net.	0	IN	A	0.0.0.0

Effectively the client does not even try to establish a connection for the requested website, so no need for pixel server etc. Both blocking variants are covered by the report engine. Selective whitelisting (you whitelist a subdomain, even if the top level domain is blocked) does not work with the 'null' variant.

WildByDesign:

However, I would really like to understand it better before I mess around with my config.

No worries, you could simply switch between these dns varaints via LuCI.

WildByDesign:

Have you noticed better performance with dnsmasq/null?

I don't think you'll notice it when you surf. But I'm using only quite small block lists and my main router uses unbound and not dnsmasq. However the numbers of @Mushoz prove that the dnsmasq answers via "addn-hosts" are more performant ... just try it on your own.

WildByDesign:

If I were to switch to dnsmasq/null, for my second dnsmasq instance in etc/config/dhcp file, would I manually need to add a line for addnhosts method?

Yep, just add for this additional dnsmasq instance something like ...

list addnhosts '/tmp/<LISTNAME>'

but there is no need to switch both instances, you can mix both variants (nxdomain/null).

WildByDesign:

It has been a pleasure testing your program over the past 3 years.

Yep, that's really funny ... here's the first changelog from my private git ...

git-prerelease750×414 35.4 KB

BTW, just pushed adblock 3.8.1 to master, you should start with that version cause I've fixed some 'null' related things with this update (see first post).

Thanks!

RainGater · 2019-08-20T11:22:07.217Z

I don't see 3.8.1 in the snapshot branch. I see only adblock_3.8.0-1_all, same as the previous one. Is this the correct one?

Also, when going from 3.8.0-1, do we still use --force-maintainer option or not needed?

opkg install adblock_3.8.0-1_all.ipk
opkg install luci-app-adblock_git-19.230.62046-276aa89-1_all.ipk

What about cache (rm -rf /tmp/luci-*), is it required?

dibdot · 2019-08-20T11:57:14.422Z

It will take some time for the buildbots to create the new version of the package. Please wait until you'll see something like adblock_3.8.1-1[...]. If you're coming from 3.8.0 you don't need the maintainer option, nor an updated LuCI package (the frontend has not been changed).

RainGater · 2019-08-21T11:35:11.418Z

I still don't see your latest version, 3.8.1. I am not sure why the bots are not building your latest changes?

WildByDesign · 2019-08-21T13:09:31.193Z

I've been running 3.8.1 for close to 24 hours now.

Generally, I just look under https://downloads.openwrt.org/snapshots/packages/ and choose one with the most recent date changes. Since the package is for all architectures, it doesn't matter which directory it comes from.

Here is one which has the current package:
https://downloads.openwrt.org/snapshots/packages/arm_cortex-a15_neon-vfpv4/packages/

Cheers!

EDIT: Hang on a few hours, it appears that 3.8.2 has just been pushed. Check those various repos in a few hours for 3.8.2.

dibdot · 2019-08-21T13:55:32.025Z

WildByDesign:

Hang on a few hours, it appears that 3.8.2 has just been pushed

Yep, sorry for the mess - just found & fixed a few more bugs in 3.8.1 ... hopefully this one is more stable ...

RainGater · 2019-08-22T01:09:42.023Z

I selected the DNS Blocking Variant to be "null (IPv/IP6) dnsmasq only". Do I have to specify addn-hosts or anything else for it to be effective as I am not sure what that means?

I picked all the available blocklist sources, which comes out to 185933 domains. Then, I picked "null (IPv/IP6) dnsmasq only" option as well as 'DNS File Reset'. Do I have to do something with addn-hosts or it's not needed?

When I run the dns query report, it has blocked 24% of the domains:
Total DNS Queries: 10643
Blocked DNS Queries: 2562 (24.07 %)

dibdot · 2019-08-22T10:49:11.497Z

RainGater:

Do I have to specify addn-hosts or anything else for it to be effective as I am not sure what that means?

Your setup looks OK. You don't need any special/manual settings, the addnhosts entry will be done automatically by adblock (check the file /etc/config/dhcp).

One major drawback of the 'null' blocking method is, that it only works per single host entry ... you can't block a whole domain (e.g. including all subdomains) with one single entry. That's may one reason why these pihole guys working with such big blocking lists.

RainGater · 2019-08-22T13:45:20.957Z

I see the following in /etc/config/dhcp.

option serversfile '/tmp/adb_list.overall'

But, /tmp/adb_list.overall is 0 bytes as there is nothing there. What else do you want me to see in /etc/config/dhcp?

root@r7800:/tmp# ll /tmp/adb_list.overall
-rw-r--r-- 1 dnsmasq root 0 Aug 22 08:28 /tmp/adb_list.overall

dibdot · 2019-08-22T14:29:43.624Z

Then you've switched back to nxdomain?

If you select dnsmasq + nxdomain you'll find something like that in /etc/config/dhcp:

option serversfile '/tmp/adb_list.overall'

If you change to one of the 'null' variants, you'll find this line in your config:

list addnhosts '/tmp/adb_list.overall'

As long as you set the 'DNS File Reset' the produced blocklist will be purged after the DNS backend has loaded the new list ... therefore the 0 byte file is expected.