on FritzBox 4040, I have the problem and this step has not helped me to solve it
I updated the post for VPN-PBR.
I'm confused...
Is this problem due to:
- persons running
opkg upgrade
- or - is this a problem with a BusyBox bundled in a particular OpenWrt release or snapshot?
If its about opkg upgrade
- this thread needs to be added to the thread for warning about using that command.
BusyBox is breaking AdBlock, it happened to me again on my second EA8300, vgaetera's fix does not work for me, I fix with AdBlock remove and reinstall as per my prior post
I'll ask again...
- Did you actually upgrade BusyBox? - generally upgrading packages is a known issue
- If not, please provide the exact version on OpenWrt you are using.
Yes, last BusyBox upgrade to 1.30.1-6 using luci or opkg breaks AdBlock, I fixed as per my prior post.
Using OpenWRT 19.07.3 stable release, not snapshot.
OK, in the future, consider not performing such an upgrade. Use [daily] snapshots and load packages needed immediately upon install - if you truly want/need up-to-date software on the device.
and: https://openwrt.org/meta/infobox/upgrade_packages_warning
also:
This kind of problem en masse cannot be avoided with just recommendations and warnings.
The only effective way to solve it is to use foolproof design:
Limit BusyBox applets to /bin and /sbin
Same problem here - updated busybox (it is noted now, that I won't do that again..) and stubby stopped working. Took me half a day to finally fix that part (by setting tls_ca_file
to the certificates bundle file instead of the directory) to get online again. Also had issues with openvpn and therefore with vpn routing (VPN-PBR).
so far I tried: I stopped all VPN related things to get this out of the way. I tried the marked solution - which somehow removed the stubby package (?). I reinstalled adblock, libustream-mbedtls (and stubby).
Current status: stubby needs an extra restart after a reboot and I still get errors in the adblock log. But most of the sources work. I am not 100% if those errors where there before the update or if this is just a temporary problem of the selected source. Would be glad if someone could verify this.
Sun Aug 23 09:44:29 2020 user.info adblock-4.0.6[6403]: download of 'reg_de' failed, url: https://easylist-downloads.adblockplus.org/easylistgermany+easylist.txt, rule: BEGIN{FS="[|^]"}/^\|\|([[:alnum:]_-]{1,63}\.)+[[:alpha:]]+\^(\$third-party)?$/{print tolower($3)}, categories: -, rc: 4, log: Downloading 'https://easylist-downloads.adblockplus.org/easylistgermany+easylist.txt' Failed to send request: Operation not permitted
Sun Aug 23 09:44:30 2020 user.info adblock-4.0.6[6403]: download of 'reg_cn' failed, url: https://easylist-downloads.adblockplus.org/easylistchina+easylist.txt, rule: BEGIN{FS="[|^]"}/^\|\|([[:alnum:]_-]{1,63}\.)+[[:alpha:]]+\^(\$third-party)?$/{print tolower($3)}, categories: -, rc: 4, log: Downloading 'https://easylist-downloads.adblockplus.org/easylistchina+easylist.txt' Failed to send request: Operation not permitted
Sun Aug 23 09:45:54 2020 user.info adblock-4.0.6[6403]: blocklist with overall 330150 blocked domains loaded successfully (TP-Link Archer C2600, OpenWrt 19.07.3 r11063-85e04e9f46)
my config:
Status / Version enabled / 4.0.6
Blocked Domains 330.150
Active Sources adaway, adguard, andryou, anti_ad, anudeep, bitcoin, disconnect, malwaredomains, malwarelist, notracking, openphish, phishing_army, reg_cn, reg_de, smarttv, spam404, stopforumspam, utcapitole, wally3k, whocares, winhelp, winspy, youtube, yoyo
DNS Backend dnsmasq, /tmp/dnsmasq.d
Run Utils /bin/uclient-fetch, /usr/bin/awk
Run Interfaces trigger: wan, report: br-lan
Run Directories base: /tmp, backup: /tmp, report: /tmp, jail: /tmp
Run Flags backup: 1, reset: 0, flush: 0, force: 1, search: 0, report: 1, mail: 0, jail: 0
Last Run start, 2m 59s, 480/287/279, 23.08.2020 09:45:53
Run the command and post the output:
Also check:
readlink -f $(type -p sort); readlink -f $(type -p ip)
root@charon:~# readlink -f $(type -p sort); readlink -f $(type -p ip)
/usr/bin/gnu-sort
/usr/libexec/ip-full
opkg update
opkg --force-depends remove libustream-mbedtls\*
opkg --force-reinstall install libustream-mbedtls
you are onto something:
Installing libustream-mbedtls20150806 (2020-03-13-40b563b1-1) to root...
Collected errors:
* check_data_file_clashes: Package libustream-mbedtls20150806 wants to install file /lib/libustream-ssl.so
But that file is already provided by package * libustream-openssl20150806
* opkg_install_cmd: Cannot install package libustream-mbedtls20150806.
should I remove the openssl20150806 variant?
Yes.
opkg --force-depends remove libustream-mbedtls\*
did the force-depends for mbedssl as well and restarted the router - still the problem seems to persist. keep in mind - this error only comes up for those two sources - the others are fine. Is there something special about those?
Sun Aug 23 13:35:13 2020 user.info adblock-4.0.6[5056]: download of 'reg_de' failed, url: https://easylist-downloads.adblockplus.org/easylistgermany+easylist.txt, rule: BEGIN{FS="[|^]"}/^\|\|([[:alnum:]_-]{1,63}\.)+[[:alpha:]]+\^(\$third-party)?$/{print tolower($3)}, categories: -, rc: 4, log: Downloading 'https://easylist-downloads.adblockplus.org/easylistgermany+easylist.txt' Failed to send request: Operation not permitted
Sun Aug 23 13:35:13 2020 user.info adblock-4.0.6[5056]: download of 'reg_cn' failed, url: https://easylist-downloads.adblockplus.org/easylistchina+easylist.txt, rule: BEGIN{FS="[|^]"}/^\|\|([[:alnum:]_-]{1,63}\.)+[[:alpha:]]+\^(\$third-party)?$/{print tolower($3)}, categories: -, rc: 4, log: Downloading 'https://easylist-downloads.adblockplus.org/easylistchina+easylist.txt' Failed to send request: Operation not permitted
opkg update
opkg --force-reinstall install ca-bundle curl
uci set adblock.global.adb_fetchutil="curl"
uci commit adblock
/etc/init.d/adblock restart
thanks again for the fast response - sorry for keeping you waiting so long with an answer.
unfortunately - same result.
I limited the download queue to 4 in paralell - and added debug logging.
when I do a curl from the router on the given urls - they show up correctly.
the problem with youtube is new.
Mon Aug 24 19:05:16 2020 user.debug adblock-4.0.6[19922]: f_dns ::: dns: dnsmasq, dns_dir: /tmp/dnsmasq.d, dns_file: adb_list.overall, dns_user: dnsmasq, dns_inotify: 0, dns_instance: 0, backup: 1, backup_dir: /tmp, jail_dir: /tmp
Mon Aug 24 19:05:16 2020 user.debug adblock-4.0.6[19922]: f_fetch ::: fetch_util: /usr/bin/curl, fetch_parm: --connect-timeout 20 --silent --show-error --location -o
Mon Aug 24 19:05:16 2020 user.debug adblock-4.0.6[19922]: f_bgserv ::: status: stop, bg_pid: -, dns_filereset: 0, ubus_service: /etc/adblock/adblock.monitor
Mon Aug 24 19:05:16 2020 user.debug adblock-4.0.6[19922]: f_report ::: action: reload, report: 1, search: +, count: 50, process: false, print: false, dump_util: /usr/sbin/tcpdump, repdir: /tmp, repiface: br-lan, replisten: 53, repchunksize: 5, repchunkcnt: 5, bg_pid: 18610
Mon Aug 24 19:05:16 2020 user.info adblock-4.0.6[19922]: adblock instance started ::: action: reload, priority: 0, pid: 19922
Mon Aug 24 19:05:17 2020 user.debug adblock-4.0.6[19922]: f_jsnup ::: status: running, cnt: 330243, mail: 0, mail_service: /etc/adblock/adblock.mail, mail_cnt: 0, mail_pid: -
Mon Aug 24 19:05:17 2020 user.debug adblock-4.0.6[19922]: f_uci ::: config: dhcp, change:
Mon Aug 24 19:05:17 2020 user.debug adblock-4.0.6[19922]: f_uci ::: config: firewall, change:
Mon Aug 24 19:05:17 2020 user.debug adblock-4.0.6[19922]: f_temp ::: tmp_base: /tmp, tmp_dir: /tmp/tmp.JiGjAK, cores: 2, sort_options: --temporary-directory=/tmp/tmp.JiGjAK --compress-program=gzip --batch-size=32 --parallel=2, pid_file: /var/run/adblock.pid
Mon Aug 24 19:05:17 2020 user.debug adblock-4.0.6[19922]: f_main ::: memory: 480/265/257, max_queue: 4, safe_search: 0, force_dns: 1, awk: /usr/bin/awk
Mon Aug 24 19:05:17 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: blacklist, mode: blacklist, cnt: 97, in_rc: 0, out_rc: 0
Mon Aug 24 19:05:17 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: whitelist, mode: whitelist, cnt: 18, in_rc: 0, out_rc: 0
Mon Aug 24 19:05:25 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: adaway, mode: download, cnt: 7924, in_rc: 0, out_rc: 0
Mon Aug 24 19:05:25 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: adaway, mode: backup, cnt: 7924, in_rc: 0, out_rc: 0
Mon Aug 24 19:05:34 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: adguard, mode: download, cnt: 35882, in_rc: 0, out_rc: 0
Mon Aug 24 19:05:34 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: adguard, mode: backup, cnt: 35882, in_rc: 0, out_rc: 0
Mon Aug 24 19:05:36 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: anti_ad, mode: download, cnt: 41474, in_rc: 0, out_rc: 0
Mon Aug 24 19:05:36 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: anti_ad, mode: backup, cnt: 41474, in_rc: 0, out_rc: 0
Mon Aug 24 19:05:41 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: andryou, mode: download, cnt: 67422, in_rc: 0, out_rc: 0
Mon Aug 24 19:05:41 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: andryou, mode: backup, cnt: 67422, in_rc: 0, out_rc: 0
Mon Aug 24 19:05:42 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: bitcoin, mode: download, cnt: 687, in_rc: 0, out_rc: 0
Mon Aug 24 19:05:42 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: bitcoin, mode: backup, cnt: 687, in_rc: 0, out_rc: 0
Mon Aug 24 19:05:45 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: disconnect, mode: download, cnt: 2728, in_rc: 0, out_rc: 0
Mon Aug 24 19:05:45 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: disconnect, mode: backup, cnt: 2728, in_rc: 0, out_rc: 0
Mon Aug 24 19:05:53 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: malwaredomains, mode: download, cnt: 26846, in_rc: 0, out_rc: 0
Mon Aug 24 19:05:53 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: malwaredomains, mode: backup, cnt: 26846, in_rc: 0, out_rc: 0
Mon Aug 24 19:05:56 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: anudeep, mode: download, cnt: 42380, in_rc: 0, out_rc: 0
Mon Aug 24 19:05:56 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: anudeep, mode: backup, cnt: 42380, in_rc: 0, out_rc: 0
Mon Aug 24 19:05:58 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: malwarelist, mode: download, cnt: 1104, in_rc: 0, out_rc: 0
Mon Aug 24 19:05:58 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: malwarelist, mode: backup, cnt: 1104, in_rc: 0, out_rc: 0
Mon Aug 24 19:05:58 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: openphish, mode: download, cnt: 1502, in_rc: 0, out_rc: 0
Mon Aug 24 19:05:58 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: openphish, mode: backup, cnt: 1502, in_rc: 0, out_rc: 0
Mon Aug 24 19:06:01 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: phishing_army, mode: download, cnt: 16211, in_rc: 0, out_rc: 0
Mon Aug 24 19:06:01 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: phishing_army, mode: backup, cnt: 16211, in_rc: 0, out_rc: 0
Mon Aug 24 19:06:28 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: notracking, mode: download, cnt: 218195, in_rc: 0, out_rc: 0
Mon Aug 24 19:06:29 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: notracking, mode: backup, cnt: 218195, in_rc: 0, out_rc: 0
Mon Aug 24 19:06:30 2020 user.info adblock-4.0.6[19922]: download of 'reg_de' failed, url: https://easylist-downloads.adblockplus.org/easylistgermany+easylist.txt, rule: BEGIN{FS="[|^]"}/^\|\|([[:alnum:]_-]{1,63}\.)+[[:alpha:]]+\^(\$third-party)?$/{print tolower($3)}, categories: -, rc: 7, log: curl: (7) Error
Mon Aug 24 19:06:30 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: reg_de, mode: restore, cnt: 0, in_rc: 7, out_rc: 4
Mon Aug 24 19:06:30 2020 user.info adblock-4.0.6[19922]: download of 'reg_cn' failed, url: https://easylist-downloads.adblockplus.org/easylistchina+easylist.txt, rule: BEGIN{FS="[|^]"}/^\|\|([[:alnum:]_-]{1,63}\.)+[[:alpha:]]+\^(\$third-party)?$/{print tolower($3)}, categories: -, rc: 7, log: curl: (7) Error
Mon Aug 24 19:06:30 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: reg_cn, mode: restore, cnt: 0, in_rc: 7, out_rc: 4
Mon Aug 24 19:06:31 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: smarttv, mode: download, cnt: 204, in_rc: 0, out_rc: 0
Mon Aug 24 19:06:31 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: smarttv, mode: backup, cnt: 204, in_rc: 0, out_rc: 0
Mon Aug 24 19:06:32 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: spam404, mode: download, cnt: 7064, in_rc: 0, out_rc: 0
Mon Aug 24 19:06:32 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: spam404, mode: backup, cnt: 7064, in_rc: 0, out_rc: 0
Mon Aug 24 19:06:34 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: wally3k, mode: download, cnt: 770, in_rc: 0, out_rc: 0
Mon Aug 24 19:06:34 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: wally3k, mode: backup, cnt: 770, in_rc: 0, out_rc: 0
Mon Aug 24 19:06:37 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: stopforumspam, mode: download, cnt: 10892, in_rc: 0, out_rc: 0
Mon Aug 24 19:06:37 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: stopforumspam, mode: backup, cnt: 10892, in_rc: 0, out_rc: 0
Mon Aug 24 19:06:43 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: winhelp, mode: download, cnt: 9553, in_rc: 0, out_rc: 0
Mon Aug 24 19:06:43 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: winhelp, mode: backup, cnt: 9553, in_rc: 0, out_rc: 0
Mon Aug 24 19:06:45 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: whocares, mode: download, cnt: 14452, in_rc: 0, out_rc: 0
Mon Aug 24 19:06:45 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: whocares, mode: backup, cnt: 14452, in_rc: 0, out_rc: 0
Mon Aug 24 19:07:18 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: utcapitole, mode: download, cnt: 132506, in_rc: 0, out_rc: 0
Mon Aug 24 19:07:18 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: utcapitole, mode: backup, cnt: 132506, in_rc: 0, out_rc: 0
Mon Aug 24 19:07:19 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: winspy, mode: download, cnt: 323, in_rc: 0, out_rc: 0
Mon Aug 24 19:07:19 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: winspy, mode: backup, cnt: 323, in_rc: 0, out_rc: 0
Mon Aug 24 19:07:20 2020 user.info adblock-4.0.6[19922]: preparation of 'youtube' failed, rc: 0
Mon Aug 24 19:07:20 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: youtube, mode: restore, cnt: 0, in_rc: 0, out_rc: 4
Mon Aug 24 19:07:22 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: yoyo, mode: download, cnt: 3524, in_rc: 0, out_rc: 0
Mon Aug 24 19:07:22 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: yoyo, mode: backup, cnt: 3524, in_rc: 0, out_rc: 0
Mon Aug 24 19:07:27 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: -, mode: merge, cnt: 402820, in_rc: 4, out_rc: 0
Mon Aug 24 19:07:43 2020 user.debug adblock-4.0.6[19922]: f_tld ::: source: /tmp/tmp.JiGjAK/adb_list.overall, cnt: 402820, cnt_tld: 330243
Mon Aug 24 19:07:54 2020 user.debug adblock-4.0.6[19922]: f_list ::: name: -, mode: final, cnt: 330243, in_rc: 4, out_rc: 0
Mon Aug 24 19:07:58 2020 user.debug adblock-4.0.6[19922]: f_dnsup ::: lookup_util: /usr/bin/nslookup, lookup_domain: ip.icb.at, restart_rc: 0, dns_timeout: 20, dns_cnt: 2, in_rc: 0, out_rc: 0
Mon Aug 24 19:07:58 2020 user.debug adblock-4.0.6[19922]: f_jsnup ::: status: enabled, cnt: 330243, mail: 0, mail_service: /etc/adblock/adblock.mail, mail_cnt: 0, mail_pid: -
Mon Aug 24 19:07:58 2020 user.info adblock-4.0.6[19922]: blocklist with overall 330243 blocked domains loaded successfully (TP-Link Archer C2600, OpenWrt 19.07.3 r11063-85e04e9f46)
Mon Aug 24 19:07:58 2020 user.debug adblock-4.0.6[19922]: f_rmtemp ::: tmp_dir: /tmp/tmp.JiGjAK, src_file: /tmp/adb_sources.json, pid_file: /var/run/adblock.pid
Check if downloading manually works:
curl https://easylist-downloads.adblockplus.org/easylistgermany+easylist.txt | wc -l
curl https://easylist-downloads.adblockplus.org/easylistchina+easylist.txt | wc -l
There's also some DNS issue, let's also check it:
head -n -0 /etc/resolv.* /tmp/resolv.*
nslookup ip.icb.at
nslookup ip.icb.at 8.8.8.8
ok. this is strange - i just typed that curl request which worked out fine and showed me the whole content. but if I do it again - I only get the curl (7) error
root@charon:~# curl -v "https://easylist-downloads.adblockplus.org/easylistgermany+easylist.txt"
curl: (7) Error
root@charon:~# nslookup ip.icb.at
Server: 127.0.0.1
Address: 127.0.0.1#53
Name: ip.icb.at
ip.icb.at canonical name = webtoaster.icb.at
Name: webtoaster.icb.at
Address 1: 195.230.168.88
ip.icb.at canonical name = webtoaster.icb.at
root@charon:~# nslookup ip.icb.at 8.8.8.8
Server: 8.8.8.8
Address: 8.8.8.8#53
Name: ip.icb.at
ip.icb.at canonical name = webtoaster.icb.at
Name: webtoaster.icb.at
Address 1: 195.230.168.88
ip.icb.at canonical name = webtoaster.icb.at
nslookup easylist-downloads.adblockplus.org
ping -w 3 easylist-downloads.adblockplus.org
Perhaps some of your upstream resolvers are returning incorrect replies.
It can also be some sort of connectivity related issue.