Adblock support thread

Community Builds, Projects & Packages
2205

Hi there,

I've been having issues with OISD (basic or full same results).

OpenWRT Version: 21.02.3
Adblock Version: 4.1.3

Below is log if using curl

Thu Oct 20 07:48:56 2022 user.info adblock-4.1.3[2368]: preparation of 'oisd_full' failed, rc: 0

Below is log if using uclient-fetch

Thu Oct 20 07:53:18 2022 user.info adblock-4.1.3[11067]: download of 'oisd_full' failed, url: https://dbl.oisd.nl, rule: /^([[:alnum:]_-]{1,63}\.)+[[:alpha:]]+([[:space:]]|$)/{print tolower($1)}, categories: -, rc: 8, log: Downloading 'https://dbl.oisd.nl' Connecting to 2606:4700:3037::6815:3065:443 HTTP error 403

I'm not sure what is happening. Is there a way to get more logging?

2206

I've manually updated to 4.1.4 but still the same issue with OISD, full logs below

Thu Oct 20 08:19:04 2022 user.debug adblock-4.1.4[19824]: f_rmtemp ::: tmp_dir: /tmp/tmp.FAEKoF, src_file: /tmp/adb_sources.json, pid_file: /var/run/adblock.pid
Thu Oct 20 08:23:52 2022 user.debug adblock-4.1.4[30263]: f_dns    ::: dns: dnsmasq, dns_dir: /tmp/dnsmasq.d, dns_file: adb_list.overall, dns_user: dnsmasq, dns_instance: 0, backup: 1, backup_dir: /mnt/sda1/openwrt/adblock, jail_dir: /tmp
Thu Oct 20 08:23:52 2022 user.debug adblock-4.1.4[30263]: f_fetch  ::: fetch_util: /bin/uclient-fetch, fetch_parm:  --timeout=20 -O
Thu Oct 20 08:23:52 2022 user.info adblock-4.1.4[30263]: adblock instance started ::: action: reload, priority: 0, pid: 30263
Thu Oct 20 08:23:53 2022 user.debug adblock-4.1.4[30263]: f_jsnup  ::: status: running, cnt: 170796, mail: 0, mail_service: /etc/adblock/adblock.mail, mail_cnt: 0, mail_pid: -
Thu Oct 20 08:23:53 2022 user.debug adblock-4.1.4[30263]: f_uci    ::: config: dhcp, change:
Thu Oct 20 08:23:53 2022 user.debug adblock-4.1.4[30263]: f_uci    ::: config: firewall, change:
Thu Oct 20 08:23:53 2022 user.debug adblock-4.1.4[30263]: f_temp   ::: tmp_base: /tmp, tmp_dir: /tmp/tmp.NEPmCF, sort_options: --temporary-directory=/tmp/tmp.NEPmCF --compress-program=gzip --parallel=4, pid_file: /var/run/adblock.pid
Thu Oct 20 08:23:53 2022 user.debug adblock-4.1.4[30263]: f_main   ::: memory: 511/334/356, cores: 4, safe_search: 0, force_dns: 1, awk: /usr/bin/awk
Thu Oct 20 08:23:53 2022 user.debug adblock-4.1.4[30263]: f_list   ::: name: iplist, mode: iplist, cnt: 0, in_rc: 0, out_rc: 0
Thu Oct 20 08:23:53 2022 user.debug adblock-4.1.4[30263]: f_list   ::: name: blacklist, mode: blacklist, cnt: 10, in_rc: 0, out_rc: 0
Thu Oct 20 08:23:53 2022 user.debug adblock-4.1.4[30263]: f_list   ::: name: whitelist, mode: whitelist, cnt: 2, in_rc: 0, out_rc: 0
Thu Oct 20 08:23:54 2022 user.debug adblock-4.1.4[30263]: f_list   ::: name: android_tracking, mode: download, cnt: 79, in_rc: 0, out_rc: 0
Thu Oct 20 08:23:54 2022 user.debug adblock-4.1.4[30263]: f_list   ::: name: android_tracking, mode: backup, cnt: 79, in_rc: 0, out_rc: 0
Thu Oct 20 08:23:58 2022 user.debug adblock-4.1.4[30263]: f_list   ::: name: adaway, mode: download, cnt: 7352, in_rc: 0, out_rc: 0
Thu Oct 20 08:23:58 2022 user.debug adblock-4.1.4[30263]: f_list   ::: name: adaway, mode: backup, cnt: 7352, in_rc: 0, out_rc: 0
Thu Oct 20 08:24:09 2022 user.debug adblock-4.1.4[30263]: f_list   ::: name: adguard_tracking, mode: download, cnt: 37996, in_rc: 0, out_rc: 0
Thu Oct 20 08:24:09 2022 user.debug adblock-4.1.4[30263]: f_list   ::: name: adguard_tracking, mode: backup, cnt: 37996, in_rc: 0, out_rc: 0
Thu Oct 20 08:24:10 2022 user.debug adblock-4.1.4[30263]: f_list   ::: name: adguard, mode: download, cnt: 48031, in_rc: 0, out_rc: 0
Thu Oct 20 08:24:10 2022 user.debug adblock-4.1.4[30263]: f_list   ::: name: adguard, mode: backup, cnt: 48031, in_rc: 0, out_rc: 0
Thu Oct 20 08:24:11 2022 user.debug adblock-4.1.4[30263]: f_list   ::: name: winspy, mode: download, cnt: 347, in_rc: 0, out_rc: 0
Thu Oct 20 08:24:11 2022 user.debug adblock-4.1.4[30263]: f_list   ::: name: winspy, mode: backup, cnt: 347, in_rc: 0, out_rc: 0
Thu Oct 20 08:24:11 2022 user.debug adblock-4.1.4[30263]: f_list   ::: name: doh_blocklist, mode: download, cnt: 331, in_rc: 0, out_rc: 0
Thu Oct 20 08:24:11 2022 user.debug adblock-4.1.4[30263]: f_list   ::: name: doh_blocklist, mode: backup, cnt: 331, in_rc: 0, out_rc: 0
Thu Oct 20 08:24:12 2022 user.info adblock-4.1.4[30263]: download of 'oisd_full' failed, url: https://dblw.oisd.nl, rule: /^\*\.([[:alnum:]_-]{1,63}\.)+[[:alpha:]]+([[:space:]]|$)/{print tolower($1)}, categories: -, rc: 8, log: Downloading 'https://dblw.oisd.nl' Connecting to 2606:4700:3036::ac43:b7cb:443 HTTP error 403
Thu Oct 20 08:24:12 2022 user.debug adblock-4.1.4[30263]: f_list   ::: name: oisd_full, mode: restore, cnt: 0, in_rc: 8, out_rc: 4
Thu Oct 20 08:24:40 2022 user.debug adblock-4.1.4[30263]: f_list   ::: name: stevenblack, mode: download, cnt: 203347, in_rc: 0, out_rc: 0
Thu Oct 20 08:24:40 2022 user.debug adblock-4.1.4[30263]: f_list   ::: name: stevenblack, mode: backup, cnt: 203347, in_rc: 0, out_rc: 0
Thu Oct 20 08:24:43 2022 user.debug adblock-4.1.4[30263]: f_list   ::: name: yoyo, mode: download, cnt: 3672, in_rc: 0, out_rc: 0
Thu Oct 20 08:24:43 2022 user.debug adblock-4.1.4[30263]: f_list   ::: name: yoyo, mode: backup, cnt: 3672, in_rc: 0, out_rc: 0
Thu Oct 20 08:24:46 2022 user.debug adblock-4.1.4[30263]: f_list   ::: name: -, mode: merge, cnt: 271886, in_rc: 4, out_rc: 0
Thu Oct 20 08:25:04 2022 user.debug adblock-4.1.4[30263]: f_tld    ::: source: /tmp/tmp.NEPmCF/adb_list.overall, cnt: 271886, cnt_tld: 180810
Thu Oct 20 08:25:13 2022 user.debug adblock-4.1.4[30263]: f_list   ::: name: -, mode: final, cnt: 180810, in_rc: 4, out_rc: 0
Thu Oct 20 08:25:15 2022 user.debug adblock-4.1.4[30263]: f_dnsup  ::: dns: dnsmasq, cache_cmd: -, lookup_cmd: /usr/bin/nslookup, lookup_domain: example.com, restart_rc: 0, dns_flush: 0, dns_timeout: 20, dns_cnt: 0, in_rc: 0, out_rc: 0
Thu Oct 20 08:25:15 2022 user.debug adblock-4.1.4[30263]: f_jsnup  ::: status: enabled, cnt: 180810, mail: 0, mail_service: /etc/adblock/adblock.mail, mail_cnt: 0, mail_pid: -
Thu Oct 20 08:25:15 2022 user.info adblock-4.1.4[30263]: blocklist with overall 180810 blocked domains loaded successfully (Newifi-D2, OpenWrt 21.02.3 r16554-1d4dea6d4f)
Thu Oct 20 08:25:15 2022 user.debug adblock-4.1.4[30263]: f_rmtemp ::: tmp_dir: /tmp/tmp.NEPmCF, src_file: /tmp/adb_sources.json, pid_file: /var/run/adblock.pid

By the way @dibdot, is this the latest version adblock_4.1.4-5_all.ipk? In your pinned post, it says latest is 4.1.5 but I only see that file in the release.

2207

Says access to the resource oisd_full is forbidden. The server understands the request but is refusing to fulfill it.

That's a very big list iirc, and you already have 180810 as it is. Can you accommodate the `oisd_full list?

2208

I've tried both oisd_basic and oisd_full both returning error via adblock.

Update: After re-testing, it does return 403 Forbidden status on all OISD links when trying via my routers (main and secondary/repeater). But on my local computers, I can download/open the links (via browsers and curl and wget). So it's not actually directly an Adblock issue but somehow OpenWRT is being blocked by OISD.

Update #2: After more testing, it seems Cloudflare is blocking all request to osid.nl domain when trying from any of the OpenWRT routers that I have. But somehow, allowing my computers and devices connected to the routers to open the domain so that means my IP is not blocked. This is a little bit weird to be honest.

 curl -v https://oisd.nl/
> GET / HTTP/2
> Host: oisd.nl
> user-agent: curl/7.83.1
> accept: */*
>
< HTTP/2 403
< date: Thu, 20 Oct 2022 05:25:26 GMT
< content-type: text/plain; charset=UTF-8
< content-length: 16
< x-frame-options: SAMEORIGIN
< referrer-policy: same-origin
< cache-control: private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0
< expires: Thu, 01 Jan 1970 00:00:01 GMT
< report-to: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=81pOstuc%2BzONPB2DJY58CihTODw5j%2Brfe%2F8zN5EDWUzPcNGEI1d0%2FkuDxz0hJFbQMaUgX3EqcfwkxCSBp%2B6BwSFsNyVYmSfc13Bgyt2vckwou%2FEaWxudvM6M"}],"group":"cf-nel","max_age":604800}
< nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
< strict-transport-security: max-age=15552000; includeSubDomains; preload
< x-content-type-options: nosniff
< server: cloudflare
< cf-ray: 75cf553829d56e61-HKG
< alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400
<
error code: 1020

Anyway, moving away from Adblock support thread since it's not actually an issue with Adblock. If somebody knows why this is happening, please feel free to reply.

1 Like
2209

But many people using Adblock read this thread so if there is such issue we'd better know this and can easily find it in the dedicated thread. So I think this is the right place to announce your findings.

2210

The oisd issues (most likely server side firewall issues) are fixed with these commits:
master: https://github.com/openwrt/packages/commit/f83baf221d11781b1e50b492af2353ad7438d351
22.03: https://github.com/openwrt/packages/commit/667ba4294689a796d4bba97eece1063d4984874b

4.1.5-2 adblock packages should be available within the next few days in master/22.03

7 Likes
2211

Any chance of a fix for 21.02 as well please?
I'm experiencing exactly the same issue with oisd on 21.02

2212

I just installed Adblock on Netgear 4300V2 router, which is running as a AP. OpenWrt 22.03.1 but it doesn't seems to be working. what I have done wrong.

image
image1076×1076 53.6 KB

2213

Thanks @dibdot it was CloudFlare that is blocking the access (after doing more tests). And this is what Stephen (OISD maintainer) replied to me "Add "--compressed" to curl command. Or use github mirror.". And the option in curl did work (based on test on linux) but alas that option is not available in the curl version of OpenWRT hahaha.

2214

Enable debug logging and check the logs after doing a "reload" or "restart" of Adblock for errors.

2215

Dibdot already fixed the issue, let's just wait for it to be available. But if you can't wait for it, you can temporarily edit the adblock sources file and use the github links instead for OISD. (See the commit links in his comment.

2216

I'm always on master so no worries.

1 Like
2217

Nope, sorry. Just install the updated sources file to your router manually, e.g.:

Download https://raw.githubusercontent.com/openwrt/packages/master/net/adblock/files/adblock.sources
scp this file to /etc/adblock on your router
ssh to your router ...
rm /etc/adblock/adblock.sources.gz
gzip /etc/adblock/adblock.sources
restart adblock ...
2 Likes
2218

Actually I can't see an error in your screenshot. Anyway a few hints to get you started:

  • Start with the default blocklist setup, don't add too many sources at the beginning - esp. on an old, weak router ...
  • Set an trigger interface (usually your wan interface)
    Good luck! :slight_smile:
2219

yep, the shrunken libcurl is really embarrassing... :frowning:

2220

Router is in AP mode.

log of ad-block

Fri Oct 21 05:02:11 2022 user.info adblock-4.1.5[14776]: adblock instance started ::: action: reload, priority: 0, pid: 14776
Fri Oct 21 05:06:17 2022 user.info adblock-4.1.5[14776]: download of 'oisd_basic' failed, url: https://dblw.oisd.nl/basic/, rule: /^\*\.([[:alnum:]_-]{1,63}\.)+[[:alpha:]]+([[:space:]]|$)/{print tolower($1)}, categories: -, rc: 8, log: Downloading 'https://dblw.oisd.nl/basic/' Connecting to 172.67.183.203:443 HTTP error 403
Fri Oct 21 05:11:19 2022 user.info adblock-4.1.5[14776]: blocklist with overall 316960 blocked domains loaded successfully (Netgear WNDR4300 v2, OpenWrt 22.03.1 r19777-2853b6d652)
Fri Oct 21 05:12:06 2022 user.info adblock-4.1.5[17196]: adblock instance started ::: action: reload, priority: 0, pid: 17196
Fri Oct 21 05:12:35 2022 user.info adblock-4.1.5[17196]: download of 'oisd_basic' failed, url: https://dblw.oisd.nl/basic/, rule: /^\*\.([[:alnum:]_-]{1,63}\.)+[[:alpha:]]+([[:space:]]|$)/{print tolower($1)}, categories: -, rc: 8, log: Downloading 'https://dblw.oisd.nl/basic/' Connecting to 172.67.183.203:443 HTTP error 403
Fri Oct 21 05:13:44 2022 user.info adblock-4.1.5[17196]: blocklist with overall 69403 blocked domains loaded successfully (Netgear WNDR4300 v2, OpenWrt 22.03.1 r19777-2853b6d652)
Fri Oct 21 05:14:37 2022 user.info adblock-4.1.5[18063]: Please enable the 'DNS Report' option to use the reporting feature

but I do see the ads
image

2221

Still I don't see any adblock problems here. Just make sure that all clients make use of your adblock enabled DNS server on your AP ... see dhcp option 6 (for further reading https://openwrt.org/docs/guide-user/base-system/dhcp_configuration#dhcp_options)

2222
2223

so this might be causing an issue?

2224

You do have 69403 blocked domains, so Adblock is working. You are not pulling down the osid blocklist which the commit addresses. Wait for the commit to filter down to all platforms.

Also, follow @dibdot ’s recommendation directly above my previous post.