Thank you!
@dibdot Hi I have found a new list for you.
1 Like
Running Adblock 4.0, working like a dream now @dibdot.
Thanks for your support during the pre-release.
Didn't work for myself until I enabled verbose debug logging. Also ssh to your machine and run tcpdump to see if it runs.
Thanks! 
BTW, 4.0.1 is now in master ... 
3 Likes
Please try 4.0.1 which addresses a query related timeout issue in the web frontend.
1 Like
My router is going to have NAND failure at this rate 
hello, @dibdot after pre 3.99 in raw mode this record appears, although I have not had time to look well
Tue Mar 31 16:30:45 2020 user.err adblock-4.0.1[6423]: dns backend not found, please set 'adb_dns' manually
Tue Mar 31 16:30:45 2020 user.debug adblock-4.0.1[6423]: f_dnsup ::: lookup_util: /usr/bin/nslookup, lookup_domain: example.com, restart_rc: -, dns_timeout: 20, dns_cnt: 0, in_rc: 4, out_rc: 0
Tue Mar 31 16:30:45 2020 user.debug adblock-4.0.1[6423]: f_rmtemp ::: tmp_dir: -, src_file: /tmp/adb_sources.json, pid_file: /var/run/adblock.pid
Tue Mar 31 16:30:45 2020 user.debug adblock-4.0.1[6423]: f_rmdns ::: dns: raw, status: false, dns_dir: /etc/dnscrypt-proxy2, dns_file: adb_list.overall, rt_file: /tmp/adb_runtime.json, backup_dir: /mnt/sdb1/Adblock
Tue Mar 31 16:30:45 2020 user.debug adblock-4.0.1[6423]: f_bgserv ::: status: stop, bg_pid: -, dns_filereset: 0, ubus_service: /etc/adblock/adblock.monitor
Tue Mar 31 16:30:45 2020 user.debug adblock-4.0.1[6423]: f_jsnup ::: status: error, cnt: 0, mail: 0, mail_service: /etc/adblock/adblock.mail, mail_cnt: 0, mail_pid: -
Tue Mar 31 16:30:46 2020 user.err adblock-4.0.1[6532]: dns backend not found, please set 'adb_dns' manually
Tue Mar 31 16:30:46 2020 user.debug adblock-4.0.1[6532]: f_dnsup ::: lookup_util: /usr/bin/nslookup, lookup_domain: example.com, restart_rc: -, dns_timeout: 20, dns_cnt: 0, in_rc: 4, out_rc: 0
Tue Mar 31 16:30:46 2020 user.debug adblock-4.0.1[6532]: f_rmtemp ::: tmp_dir: -, src_file: /tmp/adb_sources.json, pid_file: /var/run/adblock.pid
Tue Mar 31 16:30:46 2020 user.debug adblock-4.0.1[6532]: f_rmdns ::: dns: raw, status: true, dns_dir: /etc/dnscrypt-proxy2, dns_file: adb_list.overall, rt_file: /tmp/adb_runtime.json, backup_dir: /mnt/sdb1/Adblock
Tue Mar 31 16:30:46 2020 user.debug adblock-4.0.1[6532]: f_bgserv ::: status: stop, bg_pid: -, dns_filereset: 0, ubus_service: /etc/adblock/adblock.monitor
Tue Mar 31 16:30:46 2020 user.debug adblock-4.0.1[6532]: f_jsnup ::: status: error, cnt: 0, mail: 0, mail_service: /etc/adblock/adblock.mail, mail_cnt: 0, mail_pid: -
Gratitude
Oh my, sorry for the mess ... that was introduced in the last pre 3.99 pre-release.
I've re-released 4.0.1 with the fix (https://github.com/openwrt/packages/commit/9d053003fdc3e431dc32565d6b1002c36d3f1f98) - only relevant for "raw mode" users.
1 Like
@dibdot as always, thanks for your excellent support.
Maybe I'm bad at searching but how do I add my own lists in adblock 4.0?
Thanks! Please check the still incomplete online readme (last paragraph " Edit, add new adblock sources:") ... if you're going to add new sources on your own, please make a copy of that file and work with that copy, cause this file will be overwritten with every adblock update. To reference your copy set the option 'adb_srcarc' which points by default to '/etc/adblock/adblock.sources.gz'.
Hope this helps!
1 Like
Perfect
Wed Apr 1 10:47:40 2020 user.info adblock-4.0.1[6844]: adblock instance started ::: action: reload, priority: 0, pid: 6844
Wed Apr 1 10:47:40 2020 user.debug adblock-4.0.1[6844]: f_jsnup ::: status: running, cnt: 177301, mail: 0, mail_service: /etc/adblock/adblock.mail, mail_cnt: 0, mail_pid: -
Wed Apr 1 10:47:40 2020 user.debug adblock-4.0.1[6844]: f_uci ::: config: firewall, change:
Wed Apr 1 10:47:40 2020 user.debug adblock-4.0.1[6844]: f_temp ::: tmp_base: /tmp, tmp_dir: /tmp/tmp.cCIOpF, cores: 2, sort_options: --temporary-directory=/tmp/tmp.cCIOpF --compress-program=gzip --batch-size=32 --parallel=2, pid_file: /var/run/adblock.pid
Wed Apr 1 10:47:40 2020 user.debug adblock-4.0.1[6844]: f_main ::: memory: 2045/423/1876, max_queue: 4, safe_search: 0, force_dns: 0, awk: /usr/bin/awk
Wed Apr 1 10:47:40 2020 user.debug adblock-4.0.1[6844]: f_list ::: name: blacklist, mode: blacklist, cnt: 0, in_rc: 0, out_rc: 0
Wed Apr 1 10:47:40 2020 user.debug adblock-4.0.1[6844]: f_list ::: name: whitelist, mode: whitelist, cnt: 14, in_rc: 0, out_rc: 0
Wed Apr 1 10:47:42 2020 user.debug adblock-4.0.1[6844]: f_list ::: name: bitcoin, mode: download, cnt: 688, in_rc: 0, out_rc: 0
Wed Apr 1 10:47:47 2020 user.debug adblock-4.0.1[6844]: f_list ::: name: adaway, mode: download, cnt: 12174, in_rc: 0, out_rc: 0
Wed Apr 1 10:47:51 2020 user.debug adblock-4.0.1[6844]: f_list ::: name: adguard, mode: download, cnt: 30250, in_rc: 0, out_rc: 0
Wed Apr 1 10:47:52 2020 user.debug adblock-4.0.1[6844]: f_list ::: name: andryou, mode: download, cnt: 32426, in_rc: 0, out_rc: 0
Wed Apr 1 10:47:53 2020 user.debug adblock-4.0.1[6844]: f_list ::: name: disconnect, mode: download, cnt: 2734, in_rc: 0, out_rc: 0
Wed Apr 1 10:48:02 2020 user.debug adblock-4.0.1[6844]: f_list ::: name: malwaredomains, mode: download, cnt: 26849, in_rc: 0, out_rc: 0
Wed Apr 1 10:48:11 2020 user.debug adblock-4.0.1[6844]: f_list ::: name: dshield, mode: download, cnt: 98410, in_rc: 0, out_rc: 0
Wed Apr 1 10:48:12 2020 user.info adblock-4.0.1[6844]: download of 'hphosts' failed, url: https://hosts-file.net/ad_servers.txt, rule: /^127\.0\.0\.1[[:space:]]+([[:alnum:]_-]+\.)+[[:alpha:]]+([[:space:]]|$)/{print tolower($2)}, categories: -, rc: 28, log: curl: (28) Operation timed out after 20001 milliseconds with 0 out of 0 bytes received
Wed Apr 1 10:48:13 2020 user.debug adblock-4.0.1[6844]: f_list ::: name: smarttv, mode: download, cnt: 196, in_rc: 0, out_rc: 0
Wed Apr 1 10:48:13 2020 user.debug adblock-4.0.1[6844]: f_list ::: name: malwarelist, mode: download, cnt: 1104, in_rc: 0, out_rc: 0
Wed Apr 1 10:48:15 2020 user.debug adblock-4.0.1[6844]: f_list ::: name: openphish, mode: download, cnt: 1843, in_rc: 0, out_rc: 0
Wed Apr 1 10:48:16 2020 user.debug adblock-4.0.1[6844]: f_list ::: name: phishing_army, mode: download, cnt: 13432, in_rc: 0, out_rc: 0
Wed Apr 1 10:48:19 2020 user.debug adblock-4.0.1[6844]: f_list ::: name: spam404, mode: download, cnt: 7064, in_rc: 0, out_rc: 0
Wed Apr 1 10:48:22 2020 user.debug adblock-4.0.1[6844]: f_list ::: name: winhelp, mode: download, cnt: 10474, in_rc: 0, out_rc: 0
Wed Apr 1 10:48:24 2020 user.debug adblock-4.0.1[6844]: f_list ::: name: whocares, mode: download, cnt: 14399, in_rc: 0, out_rc: 0
Wed Apr 1 10:48:24 2020 user.debug adblock-4.0.1[6844]: f_list ::: name: sysctl, mode: download, cnt: 20566, in_rc: 0, out_rc: 0
Wed Apr 1 10:48:26 2020 user.debug adblock-4.0.1[6844]: f_list ::: name: yoyo, mode: download, cnt: 3277, in_rc: 0, out_rc: 0
Wed Apr 1 10:48:27 2020 user.debug adblock-4.0.1[6844]: f_list ::: name: -, mode: merge, cnt: 204097, in_rc: 4, out_rc: 0
Wed Apr 1 10:48:34 2020 user.debug adblock-4.0.1[6844]: f_tld ::: source: /tmp/tmp.cCIOpF/adb_list.overall, cnt: 204097, cnt_tld: 177301
Wed Apr 1 10:48:35 2020 user.debug adblock-4.0.1[6844]: f_list ::: name: -, mode: final, cnt: 177301, in_rc: 4, out_rc: 0
Wed Apr 1 10:48:35 2020 user.debug adblock-4.0.1[6844]: f_dnsup ::: lookup_util: /usr/bin/nslookup, lookup_domain: example.com, restart_rc: -, dns_timeout: 20, dns_cnt: 0, in_rc: 0, out_rc: 0
Wed Apr 1 10:48:35 2020 user.debug adblock-4.0.1[6844]: f_jsnup ::: status: enabled, cnt: 177301, mail: 0, mail_service: /etc/adblock/adblock.mail, mail_cnt: 0, mail_pid: -
Wed Apr 1 10:48:35 2020 user.info adblock-4.0.1[6844]: blocklist with overall 177301 blocked domains loaded successfully (Globalscale Marvell ESPRESSOBin Board V7, OpenWrt SNAPSHOT r12787-af5ccfbac7)
Wed Apr 1 10:48:35 2020 user.debug adblock-4.0.1[6844]: f_rmtemp ::: tmp_dir: /tmp/tmp.cCIOpF, src_file: /tmp/adb_sources.json, pid_file: /var/run/adblock.pid
Gratitude
Thanks but is it really worth compressing the file? I feel the old way lists were handled was better, with just a "static" (at least it didn't change very often) plain text file.
@dibdot Btw, seems like https://hosts-file.net/ad_servers.txt is the wrong url? I get a not found error.
Nope, hphosts has been discontinued. See https://github.com/openwrt/packages/issues/11732 and https://forums.malwarebytes.com/topic/258056-hosts-filenet-domain-lists-are-broken-what-happened/?tab=comments#comment-1370169 for reference.
1 Like
Is there an alternative?
@dibdot With hpHosts being discontinued... https://hosts-file.net/ad_servers.txt currently redirects to https://www.malwarebytes.com/browserguardad_servers.txt for me, which displays a 404 page.
I'm currently still on version 3.8.15-1 of Adblock, so I don't yet know if newer versions deal with this any better. But Adblock attempts to parse this 404 page as if it's the expected hosts file and happily sends it off to Unbound (1.10.0-2), which ends up in a crash loop because of that.
Does the newer Adblock version improve on this? If not, could you perhaps add some protections against stuff like this in the future and maybe have it send a warning message to syslog instead?
Yep, e.g.:
Thu Apr 2 07:57:49 2020 user.info adblock-4.0.1[15032]: adblock instance started ::: action: reload, priority: 0, pid: 15032
Thu Apr 2 07:58:11 2020 user.info adblock-4.0.1[15032]: download of 'hphosts' failed, url: https://hosts-file.net/ad_servers.txt, rule: /^127\.0\.0\.1[[:space:]]+([[:alnum:]_-]+\.)+[[:alpha:]]+([[:space:]]|$)/{print tolower($2)}, categories: -, rc: 4, log: Downloading 'https://hosts-file.net/ad_servers.txt' Connecting to 3.234.198.254:443 Connection error: Connection timed out
Thu Apr 2 07:58:27 2020 user.info adblock-4.0.1[15032]: blocklist with overall 62105 blocked domains loaded successfully (UBNT-ERX, OpenWrt SNAPSHOT r12787-af5ccfbac7)
@dibdot I noticed that this error: Fri Apr 3 05:05:49 2020 daemon.err adblock.sh[18365]: sh: out of range appears in syslog when Disable DNS Restarts is enabled.
Full log with verbose debug:
Fri Apr 3 05:05:36 2020 user.debug adblock-4.0.1[18365]: f_dns ::: dns: unbound, dns_dir: /var/lib/unbound, dns_file: adb_list.overall, dns_user: unbound, dns_inotify: 1, dns_instance: 0, backup: 0, backup_dir: /tmp, jail_dir: /tmp
Fri Apr 3 05:05:36 2020 user.debug adblock-4.0.1[18365]: f_fetch ::: fetch_util: /bin/uclient-fetch, fetch_parm: --timeout=20 -O
Fri Apr 3 05:05:36 2020 user.debug adblock-4.0.1[18365]: f_bgserv ::: status: stop, bg_pid: -, dns_filereset: 0, ubus_service: /etc/adblock/adblock.monitor
Fri Apr 3 05:05:36 2020 user.debug adblock-4.0.1[18365]: f_report ::: action: reload, report: 1, search: +, count: 50, process: false, print: false, dump_util: /usr/sbin/tcpdump, repdir: /tmp, repiface: br-lan, replisten: 53, repchunksize: 1, repchunkcnt: 5, bg_pid: 5804
Fri Apr 3 05:05:36 2020 user.info adblock-4.0.1[18365]: adblock instance started ::: action: reload, priority: 0, pid: 18365
Fri Apr 3 05:05:36 2020 user.debug adblock-4.0.1[18365]: f_jsnup ::: status: running, cnt: 141014, mail: 0, mail_service: /etc/adblock/adblock.mail, mail_cnt: 0, mail_pid: -
Fri Apr 3 05:05:36 2020 user.debug adblock-4.0.1[18365]: f_uci ::: config: firewall, change:
Fri Apr 3 05:05:36 2020 user.debug adblock-4.0.1[18365]: f_temp ::: tmp_base: /tmp, tmp_dir: /tmp/tmp.JBlHNP, cores: 2, sort_options: --temporary-directory=/tmp/tmp.JBlHNP --compress-program=gzip --batch-size=32 --parallel=2, pid_file: /var/run/adblock.pid
Fri Apr 3 05:05:36 2020 user.debug adblock-4.0.1[18365]: f_main ::: memory: 510/260/251, max_queue: 4, safe_search: 0, force_dns: 1, awk: /usr/bin/awk
Fri Apr 3 05:05:36 2020 user.debug adblock-4.0.1[18365]: f_list ::: name: blacklist, mode: blacklist, cnt: 0, in_rc: 0, out_rc: 0
Fri Apr 3 05:05:36 2020 user.debug adblock-4.0.1[18365]: f_list ::: name: whitelist, mode: whitelist, cnt: 10, in_rc: 0, out_rc: 0
Fri Apr 3 05:05:37 2020 user.debug adblock-4.0.1[18365]: f_list ::: name: bitcoin, mode: download, cnt: 688, in_rc: 0, out_rc: 0
Fri Apr 3 05:05:38 2020 user.debug adblock-4.0.1[18365]: f_list ::: name: disconnect, mode: download, cnt: 2735, in_rc: 0, out_rc: 0
Fri Apr 3 05:05:39 2020 user.debug adblock-4.0.1[18365]: f_list ::: name: adaway, mode: download, cnt: 12172, in_rc: 0, out_rc: 0
Fri Apr 3 05:05:39 2020 user.debug adblock-4.0.1[18365]: f_list ::: name: adguard, mode: download, cnt: 30295, in_rc: 0, out_rc: 0
Fri Apr 3 05:05:40 2020 user.debug adblock-4.0.1[18365]: f_list ::: name: yoyo, mode: download, cnt: 3279, in_rc: 0, out_rc: 0
Fri Apr 3 05:05:43 2020 user.debug adblock-4.0.1[18365]: f_list ::: name: whocares, mode: download, cnt: 14399, in_rc: 0, out_rc: 0
Fri Apr 3 05:05:45 2020 user.debug adblock-4.0.1[18365]: f_list ::: name: dshield, mode: download, cnt: 98410, in_rc: 0, out_rc: 0
Fri Apr 3 05:05:45 2020 user.debug adblock-4.0.1[18365]: f_list ::: name: -, mode: merge, cnt: 155019, in_rc: 4, out_rc: 0
Fri Apr 3 05:05:47 2020 user.debug adblock-4.0.1[18365]: f_tld ::: source: /tmp/tmp.JBlHNP/adb_list.overall, cnt: 155019, cnt_tld: 141014
Fri Apr 3 05:05:49 2020 user.debug adblock-4.0.1[18365]: f_list ::: name: -, mode: final, cnt: 141014, in_rc: 4, out_rc: 0
Fri Apr 3 05:05:49 2020 daemon.err adblock.sh[18365]: sh: out of range
Fri Apr 3 05:05:49 2020 user.debug adblock-4.0.1[18365]: f_dnsup ::: lookup_util: /usr/bin/nslookup, lookup_domain: example.com, restart_rc: -, dns_timeout: 20, dns_cnt: 0, in_rc: 0, out_rc: 0
Fri Apr 3 05:05:49 2020 user.debug adblock-4.0.1[18365]: f_jsnup ::: status: enabled, cnt: 141014, mail: 0, mail_service: /etc/adblock/adblock.mail, mail_cnt: 0, mail_pid: -
Fri Apr 3 05:05:49 2020 user.info adblock-4.0.1[18365]: blocklist with overall 141014 blocked domains loaded successfully (Linksys WRT3200ACM, OpenWrt SNAPSHOT r12793-fcd1401700)
Fri Apr 3 05:05:49 2020 user.debug adblock-4.0.1[18365]: f_rmtemp ::: tmp_dir: /tmp/tmp.JBlHNP, src_file: /tmp/adb_sources.json, pid_file: /var/run/adblock.pid
Information that goes with that log:
Status / Version
enabled / 4.0.1
Blocked Domains
141,014
Active Sources
adaway, adguard, bitcoin, disconnect, dshield, whocares, yoyo
DNS Backend
unbound, /var/lib/unbound
Run Utils
/bin/uclient-fetch, /usr/bin/awk
Run Interfaces
trigger: wan, report: br-lan
Run Directories
base: /tmp, backup: /tmp, report: /tmp, jail: /tmp
Run Flags
backup: 0, reset: 0, flush: 0, force: 1, search: 0, report: 1, mail: 0, jail: 0
Last Run
reload, 0m 13s, 510/257/248, 03.04.2020 05:05:49
Thanks a lot! Fortunately, the mistake does not cause any further problems at this point - will be fixed in 4.0.2