I recently test 2 main routers with Adblock functions, I get different Blocked Domains Qty. and behaviours on it.
I did not do any settings change in Adblock, just enable and reload.
The newer 4607 sites group, have some blocked sites which is not included in the 72360 sites group.
Tell me why, thanks.
First thing, your run dates/time are very different (3 days difference to be in fact). Block list data changes regularly.
Second thing, enable debug and check the logs since I noticed the difference is large, there might be an error loading/downloading some of the data feeds. Logs should show if there were problems.
I've got 4 routers running adblock and they almost have the same numbers. They are reloaded not the same time but like 15-20 mins apart. I have a cron tasks that reloads the feeds everyday.
Thanks. I enabled the debug log, and got the reload errors:
Sun Jun 23 10:50:44 2024 user.info adblock-4.1.5[5364]: download of 'adaway' failed, url: https://raw.githubusercontent.com/AdAway/adaway.github.io/master/hosts.txt, rule: /^127\.0\.0\.1[[:space:]]+([[:alnum:]_-]{1,63}\.)+[[:alpha:]]+([[:space:]]|$)/{print tolower($2)}, categories: -, rc: 55, log: curl: (55) Error
Sun Jun 23 10:50:44 2024 user.info adblock-4.1.5[5364]: download of 'adguard' failed, url: https://adguardteam.github.io/AdGuardSDNSFilter/Filters/filter.txt, rule: BEGIN{FS="[/|^|\r]"}/^\|\|([[:alnum:]_-]{1,63}\.)+[[:alpha:]]+[\/\^\r]+$/{print tolower($3)}, categories: -, rc: 55, log: curl: (55) Error
Sun Jun 23 10:50:44 2024 user.info adblock-4.1.5[5364]: adblock instance started ::: action: reload, priority: 0, pid: 5364
--- here is the errors: ----
Sun Jun 23 10:50:44 2024 user.info adblock-4.1.5[5364]: download of 'adaway' failed, url: https://raw.githubusercontent.com/AdAway/adaway.github.io/master/hosts.txt, rule: /^127.0.0.1[[:space:]]+([[:alnum:]-]{1,63}.)+[[:alpha:]]+([[:space:]]|$)/{print tolower($2)}, categories: -, rc: 55, log: curl: (55) Error
Sun Jun 23 10:50:44 2024 user.info adblock-4.1.5[5364]: download of 'adguard' failed, url: https://adguardteam.github.io/AdGuardSDNSFilter/Filters/filter.txt, rule: BEGIN{FS="[/|^|\r]"}/^||([[:alnum:]-]{1,63}.)+[[:alpha:]]+[/^\r]+$/{print tolower($3)}, categories: -, rc: 55, log: curl: (55) Error
Sun Jun 23 10:50:46 2024 daemon.info dnsmasq[1]: exiting on receipt of SIGTERM
Sun Jun 23 10:50:50 2024 daemon.info dnsmasq[1]: started, version 2.90 cachesize 1000
Sun Jun 23 10:50:50 2024 daemon.info dnsmasq[1]: DNS service limited to local subnets
Sun Jun 23 10:50:50 2024 daemon.info dnsmasq[1]: compile time options: IPv6 GNU-getopt no-DBus UBus no-i18n no-IDN DHCP no-DHCPv6 no-Lua TFTP no-conntrack no-ipset no-nftset no-auth no-cryptohash no-DNSSEC no-ID loop-detect inotify dumpfile
Sun Jun 23 10:50:50 2024 daemon.info dnsmasq[1]: UBus support enabled: connected to system bus
Sun Jun 23 10:50:50 2024 daemon.info dnsmasq-dhcp[1]: DHCP, IP range 192.168.31.100 -- 192.168.31.249, lease time 12h
Sun Jun 23 10:50:50 2024 daemon.info dnsmasq[1]: using only locally-known addresses for test
Sun Jun 23 10:50:50 2024 daemon.info dnsmasq[1]: using only locally-known addresses for onion
Sun Jun 23 10:50:50 2024 daemon.info dnsmasq[1]: using only locally-known addresses for localhost
Sun Jun 23 10:50:50 2024 daemon.info dnsmasq[1]: using only locally-known addresses for local
Sun Jun 23 10:50:50 2024 daemon.info dnsmasq[1]: using only locally-known addresses for invalid
Sun Jun 23 10:50:50 2024 daemon.info dnsmasq[1]: using only locally-known addresses for bind
Sun Jun 23 10:50:50 2024 daemon.info dnsmasq[1]: using only locally-known addresses for adsrv.iol.co.za
Sun Jun 23 10:50:50 2024 daemon.info dnsmasq[1]: using only locally-known addresses for banner.img.co.za
Sun Jun 23 10:50:50 2024 daemon.info dnsmasq[1]: using standard nameservers for example.com
Sun Jun 23 10:50:50 2024 daemon.info dnsmasq[1]: using 4606 more local addresses
Sun Jun 23 10:50:50 2024 daemon.info dnsmasq[1]: reading /tmp/resolv.conf.d/resolv.conf.auto
Sun Jun 23 10:50:50 2024 daemon.info dnsmasq[1]: using nameserver 192.168.1.1#53
Sun Jun 23 10:50:50 2024 daemon.info dnsmasq[1]: using only locally-known addresses for test
Sun Jun 23 10:50:50 2024 daemon.info dnsmasq[1]: using only locally-known addresses for onion
Sun Jun 23 10:50:50 2024 daemon.info dnsmasq[1]: using only locally-known addresses for localhost
Sun Jun 23 10:50:50 2024 daemon.info dnsmasq[1]: using only locally-known addresses for local
Sun Jun 23 10:50:50 2024 daemon.info dnsmasq[1]: using only locally-known addresses for invalid
Sun Jun 23 10:50:50 2024 daemon.info dnsmasq[1]: using only locally-known addresses for bind
Sun Jun 23 10:50:50 2024 daemon.info dnsmasq[1]: using only locally-known addresses for adsrv.iol.co.za
Sun Jun 23 10:50:50 2024 daemon.info dnsmasq[1]: using only locally-known addresses for banner.img.co.za
Sun Jun 23 10:50:50 2024 daemon.info dnsmasq[1]: using standard nameservers for example.com
Sun Jun 23 10:50:50 2024 daemon.info dnsmasq[1]: using 4606 more local addresses
Sun Jun 23 10:50:50 2024 daemon.info dnsmasq[1]: read /etc/hosts - 12 names
Sun Jun 23 10:50:50 2024 daemon.info dnsmasq[1]: read /tmp/hosts/dhcp.cfg01411c - 4 names
Sun Jun 23 10:50:50 2024 daemon.info dnsmasq-dhcp[1]: read /etc/ethers - 0 addresses
Sun Jun 23 10:50:51 2024 user.info adblock-4.1.5[5364]: blocklist with overall 4607 blocked domains loaded successfully (Xiaomi Mi Router AX3000T (OpenWrt U-Boot layout), AX3000T SNAPSHOT r26709-156f5e8f48)
google search curl: (55) Error, it's it's related mbedtls in the new snapshot, I have to wait the problem resolved by Adblock owner in future.
That is probabaly related to a bug in mbedtls which has already been patched in latest main snapshot:
https://git.openwrt.org/?p=feed/packages.git;a=commit;h=d97d07c6da0d02d15496d1daf2bdb5cb941c8c43
I had the same error and rebuild curl with OpenSSL for my own build and that solved it for me
Thanks, just re-build my code, the patch is applied into curl, but the errors still there. not solved the problem.
It looks like it is working for me:
"kernel": "6.6.33",
"hostname": "DL-WRX36",
"system": "ARMv8 Processor rev 4",
"model": "Dynalink DL-WRX36",
"board_name": "dynalink,dl-wrx36",
"rootfs_type": "squashfs",
"release": {
"distribution": "OpenWrt",
"version": "SNAPSHOT",
"revision": "r26637-05aec66d53",
"target": "qualcommax/ipq807x",
"description": "OpenWrt SNAPSHOT r26637-05aec66d53"
}
But I rebuild curl with OpenSSL you could try that to eliminate if that curl bug is not resolved
Thanks. noted that CURL using mbedtls by default, I have changed it to openssl, but the the erors still not solved.
Please post the output of ...
curl --version
root@AX3000T:~# curl --version
curl 8.8.0 (aarch64-openwrt-linux-gnu) libcurl/8.8.0 mbedTLS/3.6.0 nghttp2/1.62.1
Release-Date: 2024-05-22
Protocols: file ftp ftps http https ipfs ipns mqtt
Features: alt-svc HSTS HTTP2 HTTPS-proxy IPv6 Largefile SSL threadsafe UnixSockets
root@AX3000T:~#
it's not switched to openssl, I'll recompile it.
Hi, in the current snapshot, curl-v8.8.0 + patch is not work for my AX3000T.
I have to move back, copy curl-v8.6.0 from release v23.05.3 and delete the new patch, all OK:
Information
Status / Version enabled / 4.1.5
Blocked Domains 72,429
root@AX3000T:~# curl --version
curl 8.6.0 (aarch64-openwrt-linux-gnu) libcurl/8.6.0 OpenSSL/3.0.14 nghttp2/1.62.1
Release-Date: 2024-01-31
Protocols: file ftp ftps http https ipfs ipns mqtt
Features: alt-svc HSTS HTTP2 HTTPS-proxy IPv6 Largefile SSL threadsafe UnixSockets
root@AX3000T:~#
So it is a curl problem an not an adblock problem?
Adblock uses curl to download the blocked list in openwrt. But, the list host server, failed on : Curl v8.8.0 + mbedtsl.
So, adblock owner, should improve it for openwrt snapshot version.
By default adblock uses uclient-fetch, it's your decision to use curl. Nothing to fix from adblock perspective.
OK, I mean adblock for openwrt, curl upgraded to 8.8.0 + mbedtls, makes openwrt adblock failed.
This is openwrt-adblock support thread.
Sorry I don't get it - don't expect any adblock updates due to a bug in current curl, just use another download tool.
Thanks, the problem is in the openwrt packages:
luci-app-adblock , and
adblock in feeds/packages/net
The app is using curl download by default, and not request a right "curl / wget / aria2c / uclient-fetch" properly.
As a normal users like me, we donot know much about Adblock, we rely on the default settings.
The another problem is every time reboot the router, the block-list always download failed.
I have to download it manually.
As said before - by default OpenWrt ships only "uclient-fetch" (and adblock will use that). Curl has been installed by you or you use a private build that includes curl ...
Set a "Startup Trigger interface" and raise the "Trigger Delay" if necessary.
Unfortunatly, the current luci-app-adblock is set the default uses "curl" and not work on openwrt curl default settings, that's why I'm here.
No, it's not. It uses 'uclient-fetch'. Neither 'luci-app-adblock' nor 'adblock' itself depend on any curl version. See: https://openwrt.org/packages/pkgdata/adblock and https://openwrt.org/packages/pkgdata/luci-app-adblock
Thanks.
I go into the source code details in adblock.sh
f_fetch() {
local util utils insecure cnt="0"
if [ -z "${adb_fetchutil}" ]; then
utils="aria2c curl wget uclient-fetch"
for util in ${utils}; do
if { [ "${util}" = "uclient-fetch" ] && printf "%s" "${adb_packages}" | grep -q "^libustream-"; } ||
{ [ "${util}" = "wget" ] && printf "%s" "${adb_packages}" | grep -q "^wget -"; } ||
[ "${util}" = "curl" ] || [ "${util}" = "aria2c" ]; then
if [ -x "$(command -v "${util}")" ]; then
adb_fetchutil="${util}"
uci_set adblock global adb_fetchutil "${util}"
f_uci "adblock"
break
fi
fi
done
elif [ ! -x "$(command -v "${adb_fetchutil}")" ]; then
unset adb_fetchutil
fi
case "${adb_fetchutil}" in
"aria2c")
[ "${adb_fetchinsecure}" = "1" ] && insecure="--check-certificate=false"
adb_fetchparm="${adb_fetchparm:-"${insecure} --timeout=20 --allow-overwrite=true --auto-file-renaming=false --log-level=warn --dir=/ -o"}"
;;
"curl")
[ "${adb_fetchinsecure}" = "1" ] && insecure="--insecure"
adb_fetchparm="${adb_fetchparm:-"${insecure} --connect-timeout 20 --fail --silent --show-error --location -o"}"
;;
"uclient-fetch")
[ "${adb_fetchinsecure}" = "1" ] && insecure="--no-check-certificate"
adb_fetchparm="${adb_fetchparm:-"${insecure} --timeout=20 -O"}"
;;
"wget")
[ "${adb_fetchinsecure}" = "1" ] && insecure="--no-check-certificate"
adb_fetchparm="${adb_fetchparm:-"${insecure} --no-cache --no-cookies --max-redirect=0 --timeout=20 -O"}"
;;
esac
if [ -n "${adb_fetchutil}" ] && [ -n "${adb_fetchparm}" ]; then
adb_fetchutil="$(command -v "${adb_fetchutil}")"
else
f_log "err" "download utility with SSL support not found, please install 'uclient-fetch' with a 'libustream-*' variant or another download utility like 'wget', 'curl' or 'aria2'"
fi
f_log "debug" "f_fetch ::: fetch_util: ${adb_fetchutil:-"-"}, fetch_parm: ${adb_fetchparm:-"-"}"
}
It's auto-detect available Ultility, and check the curl before uclient-fetch, no default setting.
If no curl in my build, will drop to uclient-fetch, but my build has enable the curl with mbedtls, this makes the download failed. The the app set it uses Curl, not to try uclient-fetch.........
Btw, the linked block-list server host, does not accept curl 8.8.0 + mbedtls protocol, also is its problem.
The adblock app in openwrt, should be improved to not use any non-work Ultility.