OpenWrt Forum Archive

Topic: Blocking tracking, ad, spyware sites from router

The content of this topic has been archived between 19 Apr 2018 and 7 May 2018. There are no obvious gaps in this topic, but there may still be some posts missing at the end.

The biggest reason why I chose to use the openwrt firmware was because I was looking for a way to be able to selectively block websites, mostly to be able to block tracking and advertising servers for privacy reasons.

Frankly, I am tired that I was not able to see who was collecting what, without my knowledge or approval.

I do use applications like the adblock plus, but it does not work on all internet browsers, also that devices like ipads, tables and phones on the wifi are not able able to easily block this traffic.

So, after a lot of research, I loaded the openwrt and been working on it getting my openwrt router to do the work of blocking thousands of tracking, advertising and spyware servers.

I am so happy with the results, that I wanted to share what it was that has worked so far.

I have done comparisons of speed, and by making this changes most websites load faster, but I think the websites are loading faster because background connections to 3rd party tracking servers are being rejected.

You can't block everything, but by following this steps you can block most of this unwanted traffic.

First of all, the easiest way I found to edit the files I wanted to work on was copying the files from the router to my computer, editing it and copying it back. You can always do it directly on the router and edit by using the vi editor.

If you are on a Windows machine, you are might want to use apps like WinSCP for copying and gedit to edit files, windows notepad is not a good editor for Linux files. Both are free.

First File: /etc/firewall.user (Make a copy of the file before editing)
Add this 2 lines:

iptables -t nat -I PREROUTING -p tcp --dport 53 -j REDIRECT --to-ports 53
iptables -t nat -I PREROUTING -p udp --dport 53 -j REDIRECT --to-ports 53


Second File: /etc/dnsmasq.conf (Make a copy of the file before editing)

This file blocks entire domains, many of this providers like double-click and 2o7 have thousands of prefixed and its just easier to block the entire domain instead of individual domains.

Here is a sample:

address=/2o7.net/127.0.0.1
address=/207.net/127.0.0.1
address=/doubleclick.net/127.0.0.1
address=/doubleclick.com/127.0.0.1

Then restart dnsmasq service

killall dnsmasq
/etc/init.d/S60dnsmasq

Third File: /etc/hosts (Make a copy of the file before editing)

This is better for individual servers, remember to keep the 192.168.1.1 OpenWRT and 127.0.0.1 localhost.

The file I am using as a base is the one located here http://winhelp2002.mvps.org/hosts.zip, even though the file looks good, you need to edit the ignored text that starts with #, the file did not work right right until I removed all the # notes on the file. I have a copy of the clean file but have nowhere to upload it to for you guys to get.


Here is a sample

192.168.1.1 OpenWRT
127.0.0.1  localhost

127.0.0.1  ssl.google-analytics.com
127.0.0.1  sb.google.com

(Last edited by frankhou77 on 26 Feb 2012, 17:16)

Here is a copy of my dnsmasq.conf file. Because it blocks entire domains the file is not that big, my host file on the other hand is really big, small enough to fit into my router but too big to copy/paste here, once I find a place to upload I will do that.

/etc/dnsmasq.conf

address=/2o7.net/127.0.0.1             
address=/207.net/127.0.0.1             
address=/doubleclick.net/127.0.0.1     
address=/doubleclick.com/127.0.0.1     
address=/footprint.net/127.0.0.1       
address=/ru4.com/127.0.0.1             
address=/realmedia.com/127.0.0.1       
address=/33across.com/127.0.0.1
address=/imiclk.com/127.0.0.1           
address=/adbrite.com/127.0.0.1         
address=/adchemy.com/127.0.0.1         
address=/afy11.net/127.0.0.1           
address=/admeld.com/127.0.0.1           
address=/imrworldwide.com/127.0.0.1     
address=/nielsen-online.com/127.0.0.1   
address=/revsci.net/127.0.0.1           
address=/admob.com/127.0.0.1           
address=/intellitxt.com/127.0.0.1           
address=/abmr.net/127.0.0.1             
address=/flingwebads.com/127.0.0.1     
address=/rightmedia.com/127.0.0.1       
address=/rmxads.com/127.0.0.1             
address=/yieldmanager.com/127.0.0.1       
address=/yieldmanager.net/127.0.0.1       
address=/yldmgrimg.net/127.0.0.1         
address=/adbureau.net/127.0.0.1           
address=/aquantive.com/127.0.0.1         
address=/invitemedia.com/127.0.0.1         
address=/atdmt.com/127.0.0.1               
address=/netconversions.com/127.0.0.1   
address=/2mdn.net/127.0.0.1             
address=/googleadservices.com/127.0.0.1
address=/googlesyndication.com/127.0.0.1
address=/mookie1.com/127.0.0.1         
address=/themig.com/127.0.0.1           
address=/admob.com/127.0.0.1           
address=/hitbox.com/127.0.0.1           
address=/esomniture.com/127.0.0.1       
address=/offermatica.com/127.0.0.1     
address=/omniture.com/127.0.0.1         
address=/omtrdc.net/127.0.0.1           
address=/cmcore.com/127.0.0.1           
address=/coremetrics.com/127.0.0.1       
address=/crowdscience.com/127.0.0.1       
address=/scorecardresearch.com/127.0.0.1 
address=/nexac.com/127.0.0.1             
address=/nextaction.net/127.0.0.1         
address=/adsonar.com/127.0.0.1           
address=/advertising.com/127.0.0.1 
address=/atwola.com/127.0.0.1           
address=/leadback.com/127.0.0.1         
address=/tacoda.net/127.0.0.1           
address=/quantcast.com/127.0.0.1       
address=/quantserve.com/127.0.0.1       
address=/gravity.com/127.0.0.1         
address=/fimserve.com/127.0.0.1         
address=/foxnetworks.com/127.0.0.1     
address=/myads.com/127.0.0.1           
address=/rubiconproject.com/127.0.0.1   
address=/247realmedia.com/127.0.0.1     
address=/decdna.net/127.0.0.1           
address=/decideinteractive.com/127.0.0.1
address=/pm14.com/127.0.0.1             
address=/channelintelligence.com/127.0.0.1
address=/youknowbest.com/127.0.0.1       
address=/addthis.com/127.0.0.1           
address=/addthisedge.com/127.0.0.1       
address=/corp.kaltura.com/127.0.0.1       
address=/kaltura.com/127.0.0.1           
address=/targetingmarketplace.com/127.0.0.1
address=/insightexpress.com/127.0.0.1     
address=/insightexpressai.com/127.0.0.1

(Last edited by frankhou77 on 26 Feb 2012, 17:05)

Thanks for sharing. I will try this the next time I upgrade my WGT634U with a self-built new OpenWRT firmware.

Today is the day that the new Google Policy goes into effect, this new Google Policy is why I took the time to make this work on my Open-WRT Firmware, so far both this dnsmask and the host file I have work great.

It would be foolish to assume that this stops all tracking servers, but stops most of it, and for that I am already ahead.

Still looking for a place to store the host file, current file after doing all the clean up is 470k.

If you are creating your own host file, make sure this 2 entries exist or your router might not work right.

192.168.1.1 OpenWRT
127.0.0.1  localhost

Another way to do this would be to use Privoxy, it's built to do specifically what you're doing and can even be used to squash tracking cookies. There's a package for it for OpenWRT, it works extremely well on the trunk builds.

http://www.privoxy.org/

Its much easier to manage that a huge hosts file / dnsmasq configuration. I think you'll find it more capable than simple DNS blocking.

Interesting, thanks for the info, going to try it.

Are you using it? Does it affect perfornce?

I am using it. Performance depends on the device you're using and how you have privoxy configured. In its most basic configuration there's no noticeable impact.

I have just updated my hosts and dnsmasq files.

The files have been updated, duplicated entries have been removed and by using the dnsmasq for large marketing systems the host file is now smaller.

# Updated dnsmasq.conf file - 12 12 12

address=/207.net/127.0.0.1
address=/247realmedia.com/127.0.0.1
address=/247realmedia.com/127.0.0.1
address=/2mdn.net/127.0.0.1
address=/2o7.net/127.0.0.1
address=/33across.com/127.0.0.1
address=/51yes.com/127.0.0.1
address=/abmr.net/127.0.0.1
address=/adbrite.com/127.0.0.1
address=/adbureau.net/127.0.0.1
address=/adchemy.com/127.0.0.1
address=/addthis.com/127.0.0.1
address=/addthisedge.com/127.0.0.1
address=/admeld.com/127.0.0.1
address=/admob.com/127.0.0.1
address=/adocean.pl/127.0.0.1
address=/adsonar.com/127.0.0.1
address=/advertising.com/127.0.0.1
address=/advertserve.com/127.0.0.1
address=/afy11.net/127.0.0.1
address=/aquantive.com/127.0.0.1
address=/atdmt.com/127.0.0.1
address=/atwola.com/127.0.0.1
address=/axf8.net/127.0.0.1
address=/blueseek.com/127.0.0.1
address=/bravenet.com/127.0.0.1
address=/bravenet.com/127.0.0.1
address=/channelintelligence.com/127.0.0.1
address=/cmcore.com/127.0.0.1
address=/cnzz.com/127.0.0.1
address=/coremetrics.com/127.0.0.1
address=/corp.kaltura.com/127.0.0.1
address=/cqcounter.com/127.0.0.1
address=/crowdscience.com/127.0.0.1
address=/decdna.net/127.0.0.1
address=/decideinteractive.com/127.0.0.1
address=/doubleclick.com/127.0.0.1
address=/doubleclick.net/127.0.0.1
address=/edgecastcdn.net/127.0.0.1
address=/esomniture.com/127.0.0.1
address=/fastclick.net/127.0.0.1
address=/fimserve.com/127.0.0.1
address=/flingwebads.com/127.0.0.1
address=/footprint.net/127.0.0.1
address=/foxnetworks.com/127.0.0.1
address=/googleadservices.com/127.0.0.1
address=/googlesyndication.com/127.0.0.1
address=/gravity.com/127.0.0.1
address=/hitbox.com/127.0.0.1
address=/hittail.com/127.0.0.1
address=/imiclk.com/127.0.0.1
address=/imrworldwide.com/127.0.0.1
address=/insightexpress.com/127.0.0.1
address=/insightexpressai.com/127.0.0.1
address=/intellitxt.com/127.0.0.1
address=/invitemedia.com/127.0.0.1
address=/ivwbox.de/127.0.0.1
address=/kaltura.com/127.0.0.1
address=/leadback.com/127.0.0.1
address=/mercent.com/127.0.0.1
address=/metriweb.be/127.0.0.1
address=/misstrends.com/127.0.0.1
address=/mookie1.com/127.0.0.1
address=/myads.com/127.0.0.1
address=/netconversions.com/127.0.0.1
address=/nexac.com/127.0.0.1
address=/nextaction.net/127.0.0.1
address=/nielsen-online.com/127.0.0.1
address=/nuggad.net/127.0.0.1
address=/offermatica.com/127.0.0.1
address=/omniture.com/127.0.0.1
address=/omtrdc.net/127.0.0.1
address=/paypopup.com/127.0.0.1
address=/pm14.com/127.0.0.1
address=/quantcast.com/127.0.0.1
address=/quantserve.com/127.0.0.1
address=/realmedia.com/127.0.0.1
address=/revsci.net/127.0.0.1
address=/rightmedia.com/127.0.0.1
address=/rmxads.com/127.0.0.1
address=/ru4.com/127.0.0.1
address=/rubiconproject.com/127.0.0.1
address=/scorecardresearch.com/127.0.0.1
address=/sitemeter.com/127.0.0.1
address=/smartadserver.com/127.0.0.1
address=/statcounter.com/127.0.0.1
address=/tacoda.net/127.0.0.1
address=/targetingmarketplace.com/127.0.0.1
address=/themig.com/127.0.0.1
address=/tradedoubler.com/127.0.0.1
address=/valueclick.net/127.0.0.1
address=/xiti.com/127.0.0.1
address=/yesadvertising.com/127.0.0.1
address=/yesadvertising.com/127.0.0.1
address=/yieldmanager.com/127.0.0.1
address=/yieldmanager.net/127.0.0.1
address=/yldmgrimg.net/127.0.0.1
address=/youknowbest.com/127.0.0.1
address=/zango.com/127.0.0.1
address=/zedo.com/127.0.0.1

Tried to copy/paste the updated hosts file but its too large to copy the text here.

frankhou77 wrote:

Tried to copy/paste the updated hosts file but its too large to copy the text here.

Could you please send me a copy of your host file?  Thanks,

Hi Frank,

is there a way to make these work for ipv6 tunnels? I've understood that redirect does not work for ip6tables.



iptables -t nat -I PREROUTING -p tcp --dport 53 -j REDIRECT --to-ports 53
iptables -t nat -I PREROUTING -p udp --dport 53 -j REDIRECT --to-ports 53



Thanks,


Martijn

Not at the moment, you need to wait for OpenWrt to move to Linux 3.8.x and iptables 1.4.18.

That's a lot of work to create such custom dnsmasq.conf. You can instead tell dnsmasq to refer to additional host file(s) in the config and you can use standard Unix hosts format, that native format of most anti-ads/spyware sources.

Add the following line into /etc/config/dhcp, under the section "config dnsmasq":

list addnhosts '/tmp/block.host'

Then add these to /etc/rc.local to populate the block list during startup automatically, or use your own source URLs:

(
wget -qO- http://www.mvps.org/winhelp2002/hosts.txt|grep "^127.0.0.1" > /tmp/block.host
wget -qO- http://someonewhocares.org/hosts/hosts|grep "^127.0.0.1" >> /tmp/block.host
wget -qO- http://www.malwaredomainlist.com/hostslist/hosts.txt|grep "^127.0.0.1" >> /tmp/block.host
sort /tmp/block.host|uniq -u >/tmp/sorted
mv /tmp/sorted /tmp/block.host
/etc/init.d/dnsmasq restart ) &

(Last edited by phuque99 on 2 Apr 2013, 15:05)

While Apple devices may not be able to block the traffic, AdAway and more recently AdBlock are available for Android (neither in the market anymore as Google banned them on the grounds that they interfered with the proper operation of websites). You may also want to look at the source code for these as AdAway uses several sources for domain lists and integrates them automatically with the device's hosts file.

phuque99 wrote:

That's a lot of work to create such custom dnsmasq.conf. You can instead tell dnsmasq to refer to additional host file(s) in the config and you can use standard Unix hosts format, that native format of most anti-ads/spyware sources.

Add the following line into /etc/config/dhcp, under the section "config dnsmasq":

list addnhosts '/tmp/block.host'

Then add these to /etc/rc.local to populate the block list during startup automatically, or use your own source URLs:

(
wget -qO- http://www.mvps.org/winhelp2002/hosts.txt|grep "^127.0.0.1" > /tmp/block.host
wget -qO- http://someonewhocares.org/hosts/hosts|grep "^127.0.0.1" >> /tmp/block.host
wget -qO- http://www.malwaredomainlist.com/hostslist/hosts.txt|grep "^127.0.0.1" >> /tmp/block.host
sort /tmp/block.host|uniq -u >/tmp/sorted
mv /tmp/sorted /tmp/block.host
/etc/init.d/dnsmasq restart ) &

Thanks for that!

jow wrote:

Not at the moment, you need to wait for OpenWrt to move to Linux 3.8.x and iptables 1.4.18.

Thanks Jow! Maybe you have a pointer for me where to look in case i make the jump?


Thanks

Here's an ad blocking script that works for me.  This uses an internal http server (pixelserv) to return a blank GIF for any blocked hosts.  The benefit is that it avoid's seeing a bunch of 404 file not found errors in the web browser.

Step 1) Configure a network alias that will be used for the Pixelserv http server (pick a free IP address)

uci add network alias
uci set network.@alias[-1].interface=lan
uci set network.@alias[-1].proto=static
uci set network.@alias[-1].ipaddr=192.168.0.254 (pick a free IP address on your LAN)
uci set network.@alias[-1].netmask=255.255.255.0
uci commit

2) Configure a uhttpd server that will be the pixelserv (and return a blank.gif from any request)

uci add uhttpd uhttpd
uci rename uhttpd.@uhttpd[-1]=pixelserv
uci add_list uhttpd.@uhttpd[-1].listen_http=0.0.0.0:88
uci set uhttpd.@uhttpd[-1].home=/www2
uci set uhttpd.@uhttpd[-1].rfc1918_filter=1
uci set uhttpd.@uhttpd[-1].max_requests=3
uci set uhttpd.@uhttpd[-1].error_page=/blank.gif
uci set uhttpd.@uhttpd[-1].index_page=blank.gif
uci set uhttpd.@uhttpd[-1].network_timeout=30
uci set uhttpd.@uhttpd[-1].tcp_keepalive=1
uci commit

Step 3) Configure a firewall rule to redirect requests to a blocked host to the pixelserv

uci add firewall redirect
uci set firewall.@redirect[-1].target=DNAT
uci set firewall.@redirect[-1].src=lan
uci set firewall.@redirect[-1].proto=tcp
uci set firewall.@redirect[-1].src_dip=(the alias IP address from step 1)
uci set firewall.@redirect[-1].src_dport=80
uci set firewall.@redirect[-1].dest_ip=(the alias IP address from step 1)
uci set firewall.@redirect[-1].dest_port=88
uci set firewall.@redirect[-1].name=Pixelserv
uci commit

4) Download a blank.gif to the root of the pixelserv

mkdir /www2
wget -O /www2/blank.gif http://upload.wikimedia.org/wikipedia/commons/c/c0/Blank.gif

5) Configure a startup script to download hosts to block

cat <<EOF > /etc/init.d/adblock
#!/bin/sh /etc/rc.common
START=59
start() {
pixel="\`ifconfig br-lan | grep inet | awk '{ print \$3 }' | awk -F ":" '{ print \$2 }' | cut -d . -f 1,2,3\`.254"
touch /var/adhosts
wget -O - "http://pgl.yoyo.org/adservers/serverlist.php?hostformat=dnsmasq&showintro=0&mimetype=plaintext" | sed "s/127\.0\.0\.1/\$pixel/" > /var/adhosts
}
EOF

5) Configure dnsmasq use the list of blocked addresses, which will be redirected to the pixelserv

echo "conf-file=/var/adhosts" >> /etc/dnsmasq.conf

7) Enable the startup script to automatically download the hosts to block

chmod 755 /etc/init.d/adblock
/etc/init.d/adblock enable

Dear languagegame, your system works and returns a blank page if you do not use polipo. If I use the polipo web proxy (port 8123) I see a bunch of "404 File not found". How can I fix? Thanks

(Last edited by bru2001 on 26 Apr 2013, 17:43)

languagegame wrote:

Here's an ad blocking script that works for me.  This uses an internal http server (pixelserv) to return a blank GIF for any blocked hosts.  The benefit is that it avoid's seeing a bunch of 404 file not found errors in the web browser.

It does block the ads, but still gives me the 404. Using Openwrt Attitude Adjustment 12.09.

The http-server pixelserv is up, it returns the blank GIF on the given IP, but the redirection to the pixelserv doesn't work.

Can you give further explanation to step 3?

Thanks,
GS

Very interesting languagegame
But, all I tried worked just in a semi-acceptable way. Often got errors and 404's, making the remedy worse than the disease tongue,
For now, I'm keeping with Chrome AdBlock hehe.
But still it's very nice method, redirecting  to a blank gif, I hope it will work soon!
Lately I got almost no time to spend on this things, too much work... roll

(Last edited by dabyd64 on 14 May 2013, 20:47)

Created a script "adblock", which is based on /languagegame/ ideas and using YOYO sitelist

https://gist.github.com/aarmot/5730468

Just follow instructions on top of the script.