Adblock support thread


#650

Are those blacklists really useful in this type of adblocking? They contain a lot of rule based url syntax.


#651

IMHO in most use cases not.


#652

Ok, Thank you


#653

How do I install the 3.6.0 or 3.6.1 versions if the package feeds only show 3.5.5.3?


#654

Just download the ipk package files (referenced in the first post) directly to your router and install them via opkg.


#655

Hello,

Pearhaps you also have to cleanup the cache from Luci with :

/etc/init.d/uhttpd stop
rm -f /tmp/luci-indexcache
rm -rf /tmp/luci-modulecache
/etc/init.d/uhttpd start

I tryied few day ago without cleaning the cache and LUCI was out of order.
Not knowing at the time how to cleanup the cache I reinstalled the old version (discovered set of commands by reading posts later).


#656

good point - thanks!

It's sufficient to clean up the cache, an uhttpd restart is not required, e.g. the following one liner do the job:

rm -rf /tmp/luci-*

#657

adblock 3.6.2 is now in snapshots (only the main package has been updated, no LuCI frontend changes). Main changes are:

  • enhance the query function to search in adblock backups as well,
    to get back the set of blocking lists sources for a certain domain
  • add "Latest DNS Queries" report to commandline version as well
    (already in LuCI)
  • made the tld compression (the error handling) more robust,
    remove the needless 'adb_forcesrt' option
  • removed abandoned 'feodo' list source
  • updated readme

As long as adblock backups are enabled, the domain query returns the source lists for certain domains, too - e.g.:

query

Have fun!


#658

Good job and thanks for this new release (installed on Xiaomi WiFi Router 3G).

Can you add some extra functionalities :

  • instead of diplaying client IP, show the Hostname / MAC-Address
  • add filter functionalities related to client, domain, ... on "Latest DNS Queries" to easily retrieve information (perhaps with regular expressions)
  • exlude from statitics some domains (some IOT devices frequently communicate over the web : pool.ntp.org / api.netatmo.net / api.openweathermap.org /
  • disable for x seconds / minutes

Regards


#659

Thanks for your feedback. What's the use case for that?


#660

When you want to disable for a moment bit don’t want to forget to turn adblock on again (eg spouse wants to order something from crappy site).


#661

The goal is to temporarily deactivate Adblock to be able to view/check content in certain applications/web sites.


#662

Adhoc Suspend/Resume is already implemented, e.g.:
suspend

see above. If you forget to "Resume", adblock will be re-activated with the next regular blocklist update (e.g. on start/reload/restart action)


#663

I am running adblock 3.6.2 under a davidc502 build, r9028, which also includes tcpdump 4.9.2. The statistic reporting worked great for an hour and then stopped. When I run /etc/init.d/adblock report from the CLI, I get:

:::
::: Adblock DNS-Query Report
:::
  + Start    ::: 2019-01-13, 11:13:14
  + End      ::: 2019-01-13, 12:00:05
  + Total    ::: 776
  + Blocked  ::: 134 (17.27 %)

Unfortunately, it never updates, even when I hit Refresh Report within the Luci interface. Is there a CLI command to refresh the report? Any other ideas of what I should try?

Thanks


#664
  1. enable debug in adblock
  2. start /etc/init.d/adblock report
  3. provide the logread output (logread -e "adblock")
  4. provide the output of ps | grep [t]cpdump
  5. provide the output of df /tmp

Thanks!


#665

Ok, here goes, I enabled verbose debugging and here's the output of the commands. I find it very odd there is no tcpdump process running. One wonders how I got the original info.

root@OpenWrt:~# /etc/init.d/adblock report
root@OpenWrt:~# logread -e "adblock"
Sun Jan 13 11:13:24 2019 user.info adblock-3.6.2[5262]: adblock instance started ::: action: start, priority: 0, pid: 5262
Sun Jan 13 11:14:03 2019 user.info adblock-3.6.2[5262]: blocklist with overall 1105 domains loaded successfully (Linksys WRT1900ACS, Lede SNAPSHOT r9028-75fdf3ba01)
Sun Jan 13 11:15:20 2019 user.debug adblock-3.6.2[12328]: f_report ::: action: reload, report: 0, print: false, reputil: /usr/sbin/tcpdump, repdir: /tmp, repiface: br-lan, replisten: 53, repchunksize: 1, repchunkcnt: 5, bg_pid:
Sun Jan 13 11:15:20 2019 user.info adblock-3.6.2[12328]: adblock instance started ::: action: reload, priority: 0, pid: 12328
Sun Jan 13 11:15:20 2019 user.debug adblock-3.6.2[12328]: f_jsnup  ::: status: running, mode: normal mode, cnt: 1105, notify: 0, notify_cnt: 0, notify_pid: -
Sun Jan 13 11:15:20 2019 user.debug adblock-3.6.2[12328]: f_uci    ::: config: dhcp, change:
Sun Jan 13 11:15:20 2019 user.debug adblock-3.6.2[12328]: f_uci    ::: config: firewall, change:
Sun Jan 13 11:15:20 2019 user.debug adblock-3.6.2[12328]: f_main   ::: dns: dnsmasq, fetch_util: /bin/uclient-fetch (libustream-ssl), backup: 0, backup_mode: 0, dns_jail: 0, force_dns: 0, mem_total: 510, mem_free: 389, max_queue: 4
Sun Jan 13 11:15:20 2019 user.debug adblock-3.6.2[12328]: f_tld    ::: source: /tmp/tmp.JpinoH/tmp.raw_whitelist, cnt: 2, cnt_srt: 2, cnt_tld: 2, tld_ok: true
Sun Jan 13 11:15:20 2019 user.debug adblock-3.6.2[12328]: f_main   ::: name: adaway, enabled: 1
Sun Jan 13 11:15:20 2019 user.debug adblock-3.6.2[12328]: f_main   ::: name: adguard, enabled: 1
Sun Jan 13 11:15:20 2019 user.debug adblock-3.6.2[12328]: f_main   ::: name: bitcoin, enabled: 1
Sun Jan 13 11:15:20 2019 user.debug adblock-3.6.2[12328]: f_main   ::: name: blacklist, enabled: 1
Sun Jan 13 11:15:20 2019 user.debug adblock-3.6.2[12328]: f_main   ::: name: disconnect, enabled: 1
Sun Jan 13 11:15:33 2019 user.debug adblock-3.6.2[12328]: f_main   ::: name: bitcoin, url: https://raw.githubusercontent.com/hoshsadiq/adblock-nocoin-list/master/hosts.txt, rc: 4, log: Downloading 'https://raw.githubusercontent.com/hoshsadiq/adblock-nocoin-list/master/hosts.txt' Failed to establish connection
Sun Jan 13 11:15:33 2019 user.debug adblock-3.6.2[12328]: f_main   ::: name: disconnect, url: https://s3.amazonaws.com/lists.disconnect.me/simple_malvertising.txt, rc: 4, log: Downloading 'https://s3.amazonaws.com/lists.disconnect.me/simple_malvertising.txt' Failed to establish connection
Sun Jan 13 11:15:33 2019 user.debug adblock-3.6.2[12328]: f_main   ::: name: adguard, url: https://filters.adtidy.org/windows/filters/15.txt, rc: 4, log: Downloading 'https://filters.adtidy.org/windows/filters/15.txt' Failed to establish connection
Sun Jan 13 11:15:35 2019 user.debug adblock-3.6.2[12328]: f_main   ::: name: adaway, url: https://adaway.org/hosts.txt, rc: 4, log: Downloading 'https://adaway.org/hosts.txt' Failed to establish connection
Sun Jan 13 11:15:35 2019 user.debug adblock-3.6.2[12328]: f_main   ::: name: dshield, enabled: 1
Sun Jan 13 11:15:35 2019 user.debug adblock-3.6.2[12328]: f_main   ::: name: feodo, enabled: 1
Sun Jan 13 11:15:35 2019 user.debug adblock-3.6.2[12328]: f_main   ::: name: hphosts, enabled: 1
Sun Jan 13 11:15:35 2019 user.debug adblock-3.6.2[12328]: f_main   ::: name: malware, enabled: 1
Sun Jan 13 11:15:40 2019 user.debug adblock-3.6.2[12328]: f_main   ::: name: dshield, url: https://www.dshield.org/feeds/suspiciousdomains_Low.txt, rc: 4, log: Downloading 'https://www.dshield.org/feeds/suspiciousdomains_Low.txt' Failed to establish connection
Sun Jan 13 11:15:40 2019 user.debug adblock-3.6.2[12328]: f_main   ::: name: feodo, url: https://feodotracker.abuse.ch/blocklist/?download=domainblocklist, rc: 4, log: Downloading 'https://feodotracker.abuse.ch/blocklist/?download=domainblocklist' Failed to establish connection
Sun Jan 13 11:15:49 2019 user.debug adblock-3.6.2[12328]: f_main   ::: name: malware, url: https://mirror.espoch.edu.ec/malwaredomains/justdomains, rc: 4, log: Downloading 'https://mirror.espoch.edu.ec/malwaredomains/justdomains' Failed to establish connection
Sun Jan 13 11:15:50 2019 user.debug adblock-3.6.2[12328]: f_main   ::: name: hphosts, url: https://hosts-file.net/ad_servers.txt, rc: 4, log: Downloading 'https://hosts-file.net/ad_servers.txt' Failed to establish connection
Sun Jan 13 11:15:50 2019 user.debug adblock-3.6.2[12328]: f_main   ::: name: malwarelist, enabled: 1
Sun Jan 13 11:15:50 2019 user.debug adblock-3.6.2[12328]: f_main   ::: name: openphish, enabled: 1
Sun Jan 13 11:15:50 2019 user.debug adblock-3.6.2[12328]: f_main   ::: name: ransomware, enabled: 1
Sun Jan 13 11:15:50 2019 user.debug adblock-3.6.2[12328]: f_main   ::: name: reg_cn, enabled: 0
Sun Jan 13 11:15:50 2019 user.debug adblock-3.6.2[12328]: f_list   ::: name: reg_cn, mode: remove, cnt: 0, in_rc: 0, out_rc: 0
Sun Jan 13 11:15:50 2019 user.debug adblock-3.6.2[12328]: f_main   ::: name: reg_cz, enabled: 0
Sun Jan 13 11:15:50 2019 user.debug adblock-3.6.2[12328]: f_list   ::: name: reg_cz, mode: remove, cnt: 0, in_rc: 0, out_rc: 0
Sun Jan 13 11:15:50 2019 user.debug adblock-3.6.2[12328]: f_main   ::: name: reg_de, enabled: 0
Sun Jan 13 11:15:50 2019 user.debug adblock-3.6.2[12328]: f_list   ::: name: reg_de, mode: remove, cnt: 0, in_rc: 0, out_rc: 0
Sun Jan 13 11:15:50 2019 user.debug adblock-3.6.2[12328]: f_main   ::: name: reg_id, enabled: 0
Sun Jan 13 11:15:50 2019 user.debug adblock-3.6.2[12328]: f_list   ::: name: reg_id, mode: remove, cnt: 0, in_rc: 0, out_rc: 0
Sun Jan 13 11:15:50 2019 user.debug adblock-3.6.2[12328]: f_main   ::: name: reg_nl, enabled: 0
Sun Jan 13 11:15:50 2019 user.debug adblock-3.6.2[12328]: f_list   ::: name: reg_nl, mode: remove, cnt: 0, in_rc: 0, out_rc: 0
Sun Jan 13 11:15:50 2019 user.debug adblock-3.6.2[12328]: f_main   ::: name: reg_pl, enabled: 0
Sun Jan 13 11:15:50 2019 user.debug adblock-3.6.2[12328]: f_list   ::: name: reg_pl, mode: remove, cnt: 0, in_rc: 0, out_rc: 0
Sun Jan 13 11:15:50 2019 user.debug adblock-3.6.2[12328]: f_main   ::: name: reg_ro, enabled: 0
Sun Jan 13 11:15:50 2019 user.debug adblock-3.6.2[12328]: f_list   ::: name: reg_ro, mode: remove, cnt: 0, in_rc: 0, out_rc: 0
Sun Jan 13 11:15:50 2019 user.debug adblock-3.6.2[12328]: f_main   ::: name: reg_ru, enabled: 0
Sun Jan 13 11:15:50 2019 user.debug adblock-3.6.2[12328]: f_list   ::: name: reg_ru, mode: remove, cnt: 0, in_rc: 0, out_rc: 0
Sun Jan 13 11:15:50 2019 user.debug adblock-3.6.2[12328]: f_main   ::: name: shalla, enabled: 0
Sun Jan 13 11:15:50 2019 user.debug adblock-3.6.2[12328]: f_list   ::: name: shalla, mode: remove, cnt: 0, in_rc: 0, out_rc: 0
Sun Jan 13 11:15:50 2019 user.debug adblock-3.6.2[12328]: f_main   ::: name: spam404, enabled: 1
root@OpenWrt:~# ps | grep [t]cpdump
root@OpenWrt:~# df /tmp
Filesystem           1K-blocks      Used Available Use% Mounted on
tmpfs                   255460      4828    250632   2% /tmp
root@OpenWrt:~# ls -al /tmp
drwxrwxrwt   22 root     root           640 Jan 13 11:16 .
drwxr-xr-x    1 root     root           544 Jan 13 10:59 ..
-rw-r--r--    1 root     root             0 Jan 13 11:15 .adb_list.overall
drwx------    2 root     root            40 Jan 13 11:13 .uci
-rw-r--r--    1 root     root            23 Jan 13 11:13 TZ
-rw-r--r--    1 dnsmasq  root             0 Jan 13 11:16 adb_list.overall
-rw-r--r--    1 root     root         26804 Jan 13 11:16 adb_report.pcap0
-rw-r--r--    1 root     root           313 Jan 13 11:16 adb_runtime.json
-rw-r--r--    1 root     root          1594 Dec 31  1969 board.json
drwxr-xr-x    3 root     root            60 Jan 13 11:13 cache
-rw-r--r--    1 root     root          1004 Jan 13 11:16 dhcp.leases
drwxr-xr-x    2 root     root            40 Jan 13 11:13 dnsmasq.d
drwxr-xr-x    4 root     root           160 Jan 13 11:13 etc
drwxr-xr-x    2 root     root            40 Dec 31  1969 extroot
drwxr-xr-x    2 root     root            80 Jan 13 11:13 hosts
drwxr-xr-x    6 root     root           120 Jan 13 11:13 lib
drwxr-xr-x    4 root     root           920 Jan 13 11:13 lock
drwxr-xr-x    5 root     root           160 Jan 13 11:13 log
-rw-------    1 root     root        123714 Jan 13 11:13 luci-indexcache
drwxr-xr-x    2 root     root           900 Jan 13 11:14 luci-modulecache
drwxr-xr-x    2 root     root            40 Dec 31  1969 overlay
lrwxrwxrwx    1 root     root            21 Jan 13 11:13 resolv.conf -> /tmp/resolv.conf.auto
-rw-r--r--    1 root     root            69 Jan 13 11:13 resolv.conf.auto
drwxr-xr-x    3 root     root            60 Jan 13 11:13 rrd
drwxr-xr-x   11 root     root           620 Jan 13 11:14 run
drwxrwxrwt    2 root     root            40 Dec 31  1969 shm
drwxr-xr-x    3 root     root            60 Jan 13 11:13 spool
drwxr-xr-x    2 root     root            60 Jan 13 11:13 state
drwxr-xr-x   11 root     root          1184 Jun 22  2018 syscfg
drwxr-xr-x    2 root     root            80 Dec 31  1969 sysinfo
drwxr-xr-x    2 root     root            40 Jan 13 11:13 tmp
drwx------    2 root     root            40 Jan 13 11:13 wireguard

#666

reporting is not enabled in your config (report: 0)!


#667

Thanks. But I sure didn't change that that value (I promise!). One wonders how I got the ~hour of reporting to begin with. I just enabled reporting in the Luci interface. So now I have:

Sun Jan 13 11:30:08 2019 user.debug adblock-3.6.2[19938]: f_report ::: action: report, report: 1, print: true, reputil: /usr/sbin/tcpdump, repdir: /tmp, repiface: br-lan, replisten: 53, repchunksize: 1, repchunkcnt: 5, bg_pid: 19351

So please forgive the earlier question I should have been smart enough to answer on my own. I'm going to let this run a bit and see what happens, although I'm surprised to see that I don't have a tcpdump process running:

root@OpenWrt:~# ps | grep [t]cpdump
root@OpenWrt:~#

#668

Check that pid ... that's the tcpdump background task started by adblock.


#669

With forthcoming adblock release 3.6.3, I've finished the work on the new DNS Query Report. I've added the following features:

  • the DNS Report now displays the hostname, MAC-Address or client IP (CLI & LuCI)
  • Filter the DNS Query result set for a particular domain, client or time frame (CLI & LuCI), see examples below
  • remove needless XHR.Poll-Events from Reporting page in LuCI

Examples for the new filter capabilities ...

  1. latest requests (without filter):
    filter_1

  2. latest requests (domain filter):
    filter_2

  3. latest requests (time frame filter):
    filter_3

4.latest requests (client filter):
filter_4

If nothing comes up, this will be the final version for next OpenWrt major release 19.x.

Have fun!
Dirk