which lists are enabled in your setup?
Everything except the regional ones, shalla and ut_capitole. Got plenty of RAM on this box so turned on loads.
OK, just a rough guess: disable 'urlhaus' source and see if it works again ... this source tends to include many false positives ...
Nope, that didn't sort it. In this particular instance I see NXDOMAIN being returned for 's3-eu-west-1.amazonaws.com' when I try to download a book in the Kindle app on the device and the download fails immediately after that. Whitelisting that address worked, but I'd imagine there are more servers within that cluster that may be queried for downloading of different content and from different devices 
OK, I've checked the lists and 'adguard' is the culprit. I'll have to update the filter for that list - stay tuned.
Cool, how do you tell which list has a particular host blocked?
I have a private dev version of adblock which doesn't merge the lists ... 
1 Like
For me it was 'adguard' that was blocking s3.amzonaws.com etc. Once I disabled it, I was able to download stuff like 3rd party software updates etc. 
Please test the slightly modified search regex for 'adguard' - thanks!
config source 'adguard'
option adb_src 'https://filters.adtidy.org/windows/filters/15.txt'
option adb_src_rset 'BEGIN{FS=\"[/|^|\r]\"}/^\|\|([[:alnum:]_-]+\.)+[[:alpha:]]+[\/\^\r]+$/{print tolower(\$3)}'
option adb_src_desc 'combined adguard dns filter list, frequent updates, approx. 17.000 entries'
option enabled '1'
Thanks, seems to work with patched regex.
1 Like
Excuse the relatively naive question... I've been following this thread a bit with regard to s3.amazonaws.com. Although I don't seem to have any problems, what will changing the search regex for adguard do for me? And I gather it doesn't get updated automatically when the block lists are updated?
Thanks, working great now 
1 Like
The existing adguard regex is too greedy and produce false positives - the updated regex extracts only valid ad related (sub-)domains from adguard.
Once the package has been updated, you can force a config file update via opkg update operation with the option --force-maintainer.
Ok, since this is not available to us mortals, what's my best path to determining the site name blocked for my wife's Ways app calculating alternate routes? Apparently it uses some other, blocked site, main route is OK, no alt shows up till shes out of the driveway and down the street, and her phone loses the wifi and goes to cellular.
I dont think theres a displayed site like a regular page, so this question might help others with hidden site issues.
As adblock doesn't log such things you need to get on the command line in your router and fire up tcpdump, filtering on DNS traffic and the IP address of your wife's phone. Get her to try planning a route in Waze and look for what requests are being replied to with an NXDOMAIN (non-existent domain), then try putting these domains into the whitelist, reload adblock, and try again.
An example command line would be:
root@OpenWrt:~# tcpdump -i br-lan udp port 53 and host 192.168.1.194
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br-lan, link-type EN10MB (Ethernet), capture size 262144 bytes
09:09:30.112460 IP kitchen-echo.56014 > openwrt.53: 33699+ A? as-hls-uk-live.akamaized.net. (46)
09:09:30.112776 IP openwrt.53 > kitchen-echo.56014: 33699 3/0/0 CNAME a844.w5.akamai.net., A 95.101.136.37, A 95.101.136.43 (110)
09:09:31.019867 IP kitchen-echo.35752 > openwrt.53: 36541+ A? as-hls-uk-live.akamaized.net. (46)
09:09:31.020175 IP openwrt.53 > kitchen-echo.35752: 36541 3/0/0 CNAME a844.w5.akamai.net., A 95.101.136.43, A 95.101.136.37 (110)
09:09:37.810464 IP kitchen-echo.42455 > openwrt.53: 33315+ A? as-hls-uk-live.akamaized.net. (46)
09:09:37.810780 IP openwrt.53 > kitchen-echo.42455: 33315 3/0/0 CNAME a844.w5.akamai.net., A 95.101.136.37, A 95.101.136.43 (110)
09:09:38.132844 IP kitchen-echo.53587 > openwrt.53: 63221+ A? as-hls-uk-live.akamaized.net. (46)
The 'udp port 53' part makes sure I'm only seeing DNS traffic and the 'host 192.168.1.194' filters traffic to and from the specified IP address. As you can see, the Amazon Echo in my kitchen is doing lots of look ups for akamai hosts and receiving good responses from my openwrt router.
1 Like
Regarding unknown blocked domains that you wish to determine and add to whitelist:
This is something that @dibdot sent me a while back for figuring out which domains were being blocked. I don't know how relevant this is right now at the current stage of Adblock development. So please take with a grain of salt at the moment until he (dev) can let us know if these commands are still relevant.
example to find blocked domains on certain sites for whitelisting:
1. the easy way ...
enable the network analysis builtins in chrome or firefox to identify domains
which are redirected to the adblock null-ip (default 198.18.0.1), add these domains to your whitelist
2. a bit harder ...
enable 'Log queries' in the dnsmasq configuration (via LuCI Network => DHCP/DNS),
ssh to your router and start tracing with 'logread -f -e "dnsmasq" -e "198.18.0.1"'
switch to your client, access the relevant site and check all domains
that are blocked/listed in logread, add these domains to your whitelist
=> finally restart the adblock service (/etc/init.d/adblock restart) in both variants
Point number 1, you can do in Chrome or Firefox browsers. Essentially, it involves going into the Dev Tools of the browser and looking into specific domain connections. You need to go to the Network section, refresh whatever page you were on that had problems, and check log to see which domains were blocked that may be problematic. You can then add those domains to your Adblock whitelist.
Point number 2 is a bit more involved and we may need some confirmation from developer if those commands are still relevant at this stage since that was from a couple years ago. This is what you will need to do if you have domain blockage problems on a cell phone, tablet, etc.
As I understand it, adblock doesn't return null IPs, it returns an NXDOMAIN record as that's processed much more quickly by the client. The client doesn't waste time trying to connect to 198.18.0.1 (and eventually failing), it just acknowledges that the domain doesn't exist and moves on.
The tcpdump approach described by @WiteWulf is perfectly OK. The alternatives posted by @WildByDesign referencing to an ancient/outdated adblock 1.x releases ... 
Even though slightly modified they are still valid:
-
Browser approach (hard to track!):
Open the browser dev console and look for unanswered (dns only) requests, e.g.:
-
dnsmasq approach:
Enable dnsmasq query logging in LuCI (Network=>DHCP and DNS). After that you can filter these queries in a ssh session with this filter string:
logread -fe "dnsmasq" | egrep 'query\[[A]{1,4}\]|NXDOMAIN'
output of the same site with the blocked domain:
The second approach is much more flexible, for instance you can log to an usb stick to get a longer history...
Hope this helps!
Hi, I recently started experiencing issues with some messaging apps and websites and tracked it down the adblock config file being out of date (specifically, the rules for adguard). Once I merged in all the config from the latest commit (I am using the 18.06 release) it all started working again.
The merge process is actually very tedious for a not so experienced person and it is not even obvious what needs to be done and how to troubleshoot it. Is it possible/feasible to split the config into two files: one file to enable/disable filters and the general config and the other would provide the regex rules, etc and would not even be modifiable? This way the latest changes would be automatically applied.
This would help everyone greatly.
Just wanted to say, wife was out of town at convention for several days, and couldnt look into this. And now, it looks like Ways is just flakey/slow with the alternate route query... seems not to be a blocked site after all.
That said, I gave a brief try of the method mentioned and it didnt work for me... have yet to dig back into that. I'm sure I'll be needing it someday, so I'll let you know if I can't make it work.
1 Like