OpenWrt Forum Archive

Topic: adblock package, release 2.x

The content of this topic has been archived between 22 Mar 2018 and 4 May 2018. Unfortunately there are posts – most likely complete pages – missing.

imbaSD wrote:

Update to 2.3.0-3 but still dont work

sorry, I have no idea - something has been changed ... please contact me by mail (see online doc) if you can't fix it on your own.

Hi guys,

Please be gentle, I am new to OpenWRT (coming from DD-WRT and Tomato) and still finding my way around it. I have been trying to set up adblock for a couple of days and I think I am unsuccessful. I can't find anything wrong in the logread but ads still pop up in the browsers and elsewhere as before. Am I missing a step somewhere in the set up? Restarting the service doesn't give any feedback so I have no idea if it is running or not. I read the whole thread and tried all the commands suggested and everything points to a working set up but I can't see the results.

Debug enabled. Bunch of lists enabled. Output below:

logread -e adblock
Wed Feb 15 07:44:42 2017 user.notice adblock-[2.3.0-2] debug: main    ::: dns-backend: dnsmasq, fetch-tool: /usr/bin/wget, parm: --no-config --quiet --tries=1 --no-cache --no-cookies --max-redirect=0 --timeout=5 --no-check-certificate -O
Wed Feb 15 07:44:42 2017 user.notice adblock-[2.3.0-2] debug: loop_0  ::: name: adaway, enabled: 1, dnsfile: /tmp/tmp.DKfMoo/adb_list.adaway
Wed Feb 15 07:44:45 2017 user.notice adblock-[2.3.0-2] debug: loop_1  ::: name: adaway, rc: 0
Wed Feb 15 07:44:45 2017 user.notice adblock-[2.3.0-2] debug: list    ::: name: adaway, mode: backup, rc: 0
Wed Feb 15 07:44:45 2017 user.notice adblock-[2.3.0-2] debug: loop_2  ::: name: adaway, rc: 0
Wed Feb 15 07:44:45 2017 user.notice adblock-[2.3.0-2] debug: loop_3  ::: name: adaway, rc: 0
Wed Feb 15 07:44:45 2017 user.notice adblock-[2.3.0-2] debug: loop_0  ::: name: adguard, enabled: 1, dnsfile: /tmp/tmp.DKfMoo/adb_list.adguard
Wed Feb 15 07:44:50 2017 user.notice adblock-[2.3.0-2] debug: loop_1  ::: name: adguard, rc: 0
Wed Feb 15 07:45:00 2017 user.notice adblock-[2.3.0-2] debug: list    ::: name: adguard, mode: backup, rc: 0
Wed Feb 15 07:45:00 2017 user.notice adblock-[2.3.0-2] debug: loop_2  ::: name: adguard, rc: 0
Wed Feb 15 07:45:02 2017 user.notice adblock-[2.3.0-2] debug: loop_3  ::: name: adguard, rc: 0
Wed Feb 15 07:45:02 2017 user.notice adblock-[2.3.0-2] debug: loop_0  ::: name: blacklist, enabled: 1, dnsfile: /tmp/tmp.DKfMoo/adb_list.blacklist
Wed Feb 15 07:45:02 2017 user.notice adblock-[2.3.0-2] debug: loop_1  ::: name: blacklist, rc: 0
Wed Feb 15 07:45:02 2017 user.notice adblock-[2.3.0-2] debug: list    ::: name: blacklist, mode: restore, rc: 0
Wed Feb 15 07:45:02 2017 user.notice adblock-[2.3.0-2] debug: loop_2  ::: name: blacklist, rc: 0
Wed Feb 15 07:45:02 2017 user.notice adblock-[2.3.0-2] debug: list    ::: name: blacklist, mode: remove, rc: 0
Wed Feb 15 07:45:02 2017 user.notice adblock-[2.3.0-2] debug: loop_3  ::: name: blacklist, rc: 0
Wed Feb 15 07:45:02 2017 user.notice adblock-[2.3.0-2] debug: loop_0  ::: name: disconnect, enabled: 1, dnsfile: /tmp/tmp.DKfMoo/adb_list.disconnect
Wed Feb 15 07:45:06 2017 user.notice adblock-[2.3.0-2] debug: loop_1  ::: name: disconnect, rc: 0
Wed Feb 15 07:45:07 2017 user.notice adblock-[2.3.0-2] debug: list    ::: name: disconnect, mode: backup, rc: 0
Wed Feb 15 07:45:07 2017 user.notice adblock-[2.3.0-2] debug: loop_2  ::: name: disconnect, rc: 0
Wed Feb 15 07:45:08 2017 user.notice adblock-[2.3.0-2] debug: loop_3  ::: name: disconnect, rc: 0
Wed Feb 15 07:45:08 2017 user.notice adblock-[2.3.0-2] debug: loop_0  ::: name: dshield, enabled: 1, dnsfile: /tmp/tmp.DKfMoo/adb_list.dshield
Wed Feb 15 07:45:12 2017 user.notice adblock-[2.3.0-2] debug: loop_1  ::: name: dshield, rc: 0
Wed Feb 15 07:45:13 2017 user.notice adblock-[2.3.0-2] debug: list    ::: name: dshield, mode: backup, rc: 0
Wed Feb 15 07:45:13 2017 user.notice adblock-[2.3.0-2] debug: loop_2  ::: name: dshield, rc: 0
Wed Feb 15 07:45:13 2017 user.notice adblock-[2.3.0-2] debug: loop_3  ::: name: dshield, rc: 0
Wed Feb 15 07:45:13 2017 user.notice adblock-[2.3.0-2] debug: loop_0  ::: name: feodo, enabled: 1, dnsfile: /tmp/tmp.DKfMoo/adb_list.feodo
Wed Feb 15 07:45:19 2017 user.notice adblock-[2.3.0-2] debug: loop_1  ::: name: feodo, rc: 0
Wed Feb 15 07:45:19 2017 user.notice adblock-[2.3.0-2] debug: list    ::: name: feodo, mode: restore, rc: 0
Wed Feb 15 07:45:19 2017 user.notice adblock-[2.3.0-2] debug: loop_2  ::: name: feodo, rc: 0
Wed Feb 15 07:45:19 2017 user.notice adblock-[2.3.0-2] debug: list    ::: name: feodo, mode: remove, rc: 0
Wed Feb 15 07:45:19 2017 user.notice adblock-[2.3.0-2] debug: loop_3  ::: name: feodo, rc: 0
Wed Feb 15 07:45:19 2017 user.notice adblock-[2.3.0-2] debug: loop_0  ::: name: hphosts, enabled: 1, dnsfile: /tmp/tmp.DKfMoo/adb_list.hphosts
Wed Feb 15 07:45:25 2017 user.notice adblock-[2.3.0-2] debug: loop_1  ::: name: hphosts, rc: 0
Wed Feb 15 07:45:32 2017 user.notice adblock-[2.3.0-2] debug: list    ::: name: hphosts, mode: backup, rc: 0
Wed Feb 15 07:45:32 2017 user.notice adblock-[2.3.0-2] debug: loop_2  ::: name: hphosts, rc: 0

Also, output of ubus command that I have seen recommended before:

root@OpenWrt ~# ubus call service list '{"name":"adblock_stats"}'
{
        "adblock_stats": {
                "instances": {
                        "statistics": {
                                "running": false,
                                "command": [
                                        ""
                                ],
                                "data": {
                                        "active_lists": [
                                                {
                                                        "winspy": "169",
                                                        "zeus": "414",
                                                        "adaway": "409",
                                                        "rolist": "651",
                                                        "malwarelist": "1160",
                                                        "openphish": "1261",
                                                        "ransomware": "1528",
                                                        "ruadlist": "1792",
                                                        "yoyo": "2304",
                                                        "dshield": "175",
                                                        "disconnect": "3169",
                                                        "spam404": "6257",
                                                        "adguard": "11151",
                                                        "whocares": "103",
                                                        "winhelp": "12126",
                                                        "malware": "6818",
                                                        "sysctl": "9058",
                                                        "securemecca": "6975",
                                                        "shalla": "0",
                                                        "hphosts": "0"
                                                }
                                        ],
                                        "adblock_version": "2.3.0-2",
                                        "blocked_domains": "65520",
                                        "dns_backend": "dnsmasq",
                                        "last_rundate": "15.02.2017 03:08:41",
                                        "system": "OpenWrt Designated Driver r49166"
                                }
                        }
                }
        }
}
jasenko wrote:

Hi guys,

Please be gentle, I am new to OpenWRT (coming from DD-WRT and Tomato) and still finding my way around it. I have been trying to set up adblock for a couple of days and I think I am unsuccessful. I can't find anything wrong in the logread but ads still pop up in the browsers and elsewhere as before. Am I missing a step somewhere in the set up? Restarting the service doesn't give any feedback so I have no idea if it is running or not. I read the whole thread and tried all the commands suggested and everything points to a working set up but I can't see the results.

Please try running the following commands (hopefully this helps):

opkg update
opkg install wget

As a side-note: I have manually added Steven Black's unified hosts as a source and disabled the AdAway, Malware List, whocares, winhelp, and yoyo entries as Steven Black's unified host consolidates them, in addition to a number of other lists (github{dot}com/StevenBlack/hosts).

If anyone's curious, I added the following to the adblock.conf file:

config source 'stevenblack'
    option enabled '1'
    option adb_src 'h@@ps://raw.githubusercontent{dot}com/StevenBlack/hosts/master/hosts'
    option adb_src_rset '\$0 ~/^0\.0\.0\.0[ \t]+([A-Za-z0-9_-]+\.){1,}[A-Za-z]+/{print tolower(\$2)}'
    option adb_src_desc 'unified blocklist, frequent updates, approx. 32.000 entries'

Replace {dot} above with . and @@ with tt

dibdot, I'm a long-time user and sincerely thank you for your continued and excellent development on this.

(Last edited by nd on 15 Feb 2017, 18:32)

Thanks nd for the suggestions. I will add the new source.

My version of OpeWRT is customised arokh's build. The initial version of adblock was version 1.xx and it was failing due to wget not being able to download ssl sources. So, I updated wget, updated adblock and now I can see things get downloaded.

At the same time, seems like there were major differences between the versions and some of the old files were not cleaned up. So now I have conf files in /etc/adblock folder, but also the config file /etc/config/adblock so I have no idea which config file the script is using. I will have to investigate the script a little deeper.

But I think there was some progress, last night I rebooted the router while making some other changes, and noticed that some of the ads are blocked, but only on the mobile phone. Not all wireless works, as ads are still being served on the laptop. Weird. I am hoping that the new source might help.

AFAIK arokh's adblock was set to use /etc/config/adblock, same file as @dibdot's adblock also uses, but expects different content of course. Your best bet would be to delete /etc/config/adblock and force-reinstall @dibdot's adblock (so it would install proper /etc/config/adblock).

The files in /etc/adblock likely belong to @dibdot's adblock, so they are probably correct, but if you're going to force-reinstall everything, you may be better off deleting them as well before force-reinstall.

(Last edited by stangri on 16 Feb 2017, 02:30)

stangri wrote:

AFAIK arokh's adblock was set to use /etc/config/adblock, same file as @dibdot's adblock also uses, but expects different content of course. Your best bet would be to delete /etc/config/adblock and force-reinstall @dibdot's adblock (so it would install proper /etc/config/adblock).

The files in /etc/adblock likely belong to @dibdot's adblock, so they are probably correct, but if you're going to force-reinstall everything, you may be better off deleting them as well before force-reinstall.

thanks... I tried all that and the results were the same. I can now definitely confirm that this is not being routed to the devices on the network.

Ping from the router:
root@OpenWrt ~# ping doubleclick.net
ping: bad address 'doubleclick.net'

Ping from the laptop:
ping doubleclick.net

Pinging doubleclick.net [216.58.217.14] with 32 bytes of data:
Reply from 216.58.217.14: bytes=32 time=212ms TTL=48
Reply from 216.58.217.14: bytes=32 time=194ms TTL=48
Reply from 216.58.217.14: bytes=32 time=189ms TTL=48
Reply from 216.58.217.14: bytes=32 time=216ms TTL=48

Ping statistics for 216.58.217.14:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 189ms, Maximum = 216ms, Average = 202ms

I flushed DNS cache several times, rebooted, no avail.

Do I need to create new firewall zone on the router or something. Things do appear to work on the router but no further.

jasenko wrote:

Ping from the router:
root@OpenWrt ~# ping doubleclick.net
ping: bad address 'doubleclick.net'

Ping from the laptop:
ping doubleclick.net
Pinging doubleclick.net [216.58.217.14] with 32 bytes of data:
Reply from 216.58.217.14: bytes=32 time=212ms TTL=48

Things do appear to work on the router but no further.

Are you sure that you have not configured the laptop to use Google's DNS server, or something like that?

Thanks for trying to help guys. I just couldn't figure out what was happening so I made the decision and went back to DD-WRT. It took me all of 10-15 seconds to set this up and start blocking ads on every device on the network. Life is great again. I will be missing some of the features of OpenWRT, but all the time spent on set up was starting to affect my family and my sanity :-)

I just wanted to say a quick thanks and comment on how well it works on my old Linksys E2000.  It's great peace of mind to know adverts are blocked at the front door to the internet.

Quick question:

Can you add comments to the static .blacklist file?  It would be nice to add some notes to say why I've blocked certain items.

Cheers

smed wrote:

Can you add comments to the static .blacklist file?  It would be nice to add some notes to say why I've blocked certain items.

Yes, of course. The local blacklist will be handled/parsed like the other sources, e.g. a valid /etc/adblock/adblock.blacklist with comments:

# no more alternative facts ;-)
whitehouse.gov
[...]

...same applies to whitelist, too.

Perfect.  blacklist now #commented.  Keep up the excellent work!

I've updated the first post, to summarize the latest changes.

The new "top level domain compression" in 2.4.x removes up to 40 thousand needless host entries from the block lists and lowers the memory footprint for the dns backends by 8-10 MByte.

Have fun!
Dirk

I don't seem to get the "personalized" blacklist up and running.
I can add whatever I want to the adblock.blacklist, but I can still ping the address.

> I checked the checkbox on "blacklist"
> I verified the config in /etc/config/adblock

I restarted the service, I rebooted my device
Problem remains

Any suggestions?

I didn't find yet how I can easily update adblock?
I have now version 2.3.0-2 and I know version 2.4.0 is already out there, but I have no clue how I can update from 2.3.0 to 2.4.0 (hoping that my above issue with the blacklist will disappear automagically smile )

Thanks!

icur wrote:

I don't seem to get the "personalized" blacklist up and running.
I can add whatever I want to the adblock.blacklist, but I can still ping the address.
[...]
I didn't find yet how I can easily update adblock?

Please provide full logs (with enabled debug!).
To update adblock please consult the online documentation (see first post), in short: transfer adblock package to your router and use "opkg install < package>" for installation.

I hadn't found an entry on "update".
I "reinstalled" with opkg install and now I have the latest version (2.4.0-2).

I can give you more logs, but I think this is the problem:

Sat Mar  4 07:43:02 2017 user.notice adblock-[2.4.0-2] error: required download utility with ssl support not found, e.g. install full 'wget' package
Sat Mar  4 07:43:02 2017 user.notice adblock-[2.4.0-2] error: Please check the online documentation 'h__ps://github.com/openwrt/packages/blob/master/net/adblock/files/README.md'

(Last edited by icur on 4 Mar 2017, 08:58)

I spotted this error while running adblock 2.4.0-2 just now:

Tue Mar 14 00:56:31 2017 daemon.err adblock.sh[2119]: awk: bad regex 'se[server.d9ae99824\.': Missing ']'

EDIT:

The error seems to go away when I disable the winhelp blocklist. I saved a copy of the current winhelp blocklist on Pastebin, but I'm not allowed to post links here. So, here's the Pastebin ID: 7dGezSgY

(Last edited by elxr on 14 Mar 2017, 02:02)

elxr wrote:

The error seems to go away when I disable the winhelp blocklist. I saved a copy of the current winhelp blocklist on Pastebin, but I'm not allowed to post links here. So, here's the Pastebin ID: 7dGezSgY

indeed, that was an incorrect record in the winhelp list (introduced with the last list update):

wrong: 0.0.0.0 stats.ilsemedia.nl[server down?]
correct: 0.0.0.0 stats.ilsemedia.nl #[server down?

I've already informed the author of the list and he just fixed it! smile
Anyway, the adblock "error" is harmless ... it just says that awk can't process this single record - the rest of the list is OK. I'll mute those warnings with the next release.

Edit:

Just realized that this winhelp list update contains several typos like above ... hopefully the author will fix them asap.

(Last edited by dibdot on 14 Mar 2017, 08:37)

Hello,

Where i can find latest version on 15.01?

Thx. smile

The adblock package has been added after the 15.05.1 release, so it is not available as a built .ipk for CC15.05.1. If you want it, you might install the trunk package of adblock to CC, as the package is only scripts, no binaries.

BTW, Openwrt development pretty much stopped in April/May 2016 when practically all active core developers left for LEDE - so I didn't run any CC specific tests anymore ... best is to head over to LEDE, stable release 17.01 or current LEDE snapshots. The active adblock LEDE-forum thread with download links can be found here

Is it possible to block every website other than those in whitelist? I couldn't find any solution.

Any help would be greatly appreciated.

huangjqiu wrote:

Is it possible to block every website other than those in whitelist? I couldn't find any solution.

Any help would be greatly appreciated.

Support for 'whitelist only" mode has been implemented in adblock release 3.1 ... see here

The discussion might have continued from here.