Powerful router for filtering 5000 IP addresses

Unfortunately, opkg install banip and opkg install banip return Unknown package. I guess those are not available for my platform (AR9331).

Got it! Thanks for your help.

banip is not included in 18.06. You can either download ipk and install it manually from snapshot opkg source, or just update to 19.07.

2 Likes

Done, thanks. But I get an error when enabling banIP on the config page. Will ask its developer about that.

You can consider using IPset:
https://openwrt.org/docs/guide-user/firewall/fw3_configurations/fw3_config_ipset

It doesn't need a lot of CPU power because thousands of IP can be hashed into a single set, used by one single IPtables rule. I have more than 93000 entries running on an old and "slow" Netgear WNDR3800:

~# ipset list badgeo|head -7
Name: badgeo
Type: hash:net
Revision: 6
Header: family inet hashsize 65536 maxelem 150000
Size in memory: 42872
References: 1
Number of entries: 93208
~# grep badgeo /etc/firewall.user
ipset -N badgeo hash:net maxelem 150000
iptables -A forwarding_lan_rule -m set --match-set badgeo dst -j reject
2 Likes

Wow! Ads and tracking block?

Thanks for the guide!

The /etc/firewall.user is one part of my setup (for creating ipset and associated rule). The other is in /etc/rc.local; that contains this part of the script that populates "badgeo" ipset:

hosts="
https://www.team-cymru.org/Services/Bogons/fullbogons-ipv4.txt
http://ipdeny.com/ipblocks/data/countries/{cn,ru,in,br,ve,vn,pk,sa,ir,ar,pe,co,cl,ma,bg,ro,kz,by,tr,ua,sy,ng,kp,id,bd,af,cf,cd,cu,cy,er,fj,ht,iq,lb,ly,mm,rw,so,lk,sd,ye,zw,ph}.zone
https://www.blocklist.de/downloads/export-ips_all.txt
https://www.malwaredomainlist.com/hostslist/ip.txt
https://feodotracker.abuse.ch/downloads/ipblocklist.txt
https://ransomwaretracker.abuse.ch/downloads/RW_IPBL.txt
https://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt
https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/{dshield,firehol_level1}.netset
";
for url in $hosts; do curl -s $url | egrep -o '^[0-9]{1,3}([./][0-9]{1,3}){3,4}';
done |sort -u | xargs -rn1 printf 'add badgeo %s\n' | ipset restore

Feel free to copy.

Yes; and my own host-based blocking solution is here:

However the Openwrt adblock package module is probably more user-friendly.

1 Like

The link is dead, so can be removed from there. Beside, I wouldn't trust ipdeny.com because they get the files allocated by regional registries which may block good websites with torrent related content for example.

The link is dead.
But thanks for help!

The links are not dead. The use of curly brackets {} in bash/ash shell tells the command to iterate across all "variation" inside them. The URL above will simply tell curl to run twice against both links:

https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/dshield.netset
https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/firehol_level1.netset

Thanks for the tip about ipdeny.

Got it. Thanks.

By the way, how can I add a range of IP addresses with ipset?
For example, 144.76.168.9 - 144.76.168.16

Non-CIDR or non-single IP will be a bit pain to manage. You'll need to create IPset type bitmap:ip with a specific range limit not exceeding 65535, namely /16 subnets only.

Example:

~# ipset -N testacl bitmap:ip range 144.76.0.0-144.76.255.254
~# ipset add testacl 144.76.168.9-144.76.168.16
~# ipset list testacl
Name: testacl
Type: bitmap:ip
Revision: 3
Header: range 144.76.0.0-144.76.255.254
Size in memory: 8248
References: 0
Number of entries: 8
Members:
144.76.168.9
144.76.168.10
144.76.168.11
144.76.168.12
144.76.168.13
144.76.168.14
144.76.168.15
144.76.168.16

In short, it'll be a pain in the rear if you want to use IPset with non-CIDR ranges that exists in multiple /16 subnets.

Is it only /16 that is permitted for the CIDR? I had thought that it was anything 1-32 inclusive. I also thought that the hash map could store 64k entries, each of which could be a CIDR block of any width.

A "trick" is also to consider an "exception" rule when you've got an incomplete block on a really ugly boundary

.8 -- 0b0000'1000
.9 -- 0b0000'1001

.15 -- 0b0000'1111
.16 -- 0b0001'0000

So you could write that as, for example (check this, as my coffee hasn't hit yet)

Skip 144.76.168.8/32 -- .8 (skips over blocking rules that immediately follow)
Block 144.76.168.8/29 -- .8 - .15
Block 144.76.168.16/32 -- .16

The "skip" is so that if there is a later rule that would block that IP, the packet hasn't already been accepted through this phase of your rules.

I see, thanks. I used to use a firewall software where I put the addresses I wanted to block, so I need to parse its config file with the addresses and I was going to develop a small parser for that task. But it seems like it's better to do that manually.

Just a simple question. How can you know the number of subnets, if you have just a list of addresses from the IP range?

No simple answer -- how are you collecting these 5000 IP addresses? It is remotely possible that you can use the registries to determine who has administrative authority over those IP blocks and, from that, determine the blocks themselves. Typically they are assigned (delegated) to lower-tier authorities and within organizations by a CIDR subnet -- N high-order bits match.

I used to track all those IPs during browsing websites. I defined undesired connections (ads and trackers), so decided to block them all. That's it

Then they're sort of "random samplings" of where you don't want to bother retrieving content from, rather than "hostile" IPs. In that case, you don't need to necessarily block them at the firewall.

You might want to consider a good plugin for your browser (I use uBlock Origin on my desktop systems) as the "best" option in my opinion.

Other things to explore would be DNS-based blocking with established lists, perhaps augmented by your own list. With cloud hosting, IP addresses can change very quickly, making hand-collected lists less valuable than they once were. Unless the IP lists are regularly updated, they may end up blocking "innocent" traffic. IP blocks are also problematic with multiple domains and websites being hosted on the same IP address, especially for IPv4.

The banip and adblock packages might be interesting to you to look into.

2 Likes

Thanks jeff. The fact that the IPs change frequently is the main problem here, but I'm ready to face it once an innocent service gets blocked.

I'd like to have the IPs blocked for any of my device, not just PC. I used to have a firewall installed on it, but it consumed a lot of CPU power to block that huge set of addresses. Moreover, there was an undesired system traffic that couldn't be blocked with a browser extension. So I want my portable router to do the stuff.

A few things. The first is Chaos Calmer is long past EOL and its kernel, wireless protocols, and third-party software have multiple, well-known, and actively exploited, severe security vulnerabilities. You do have to upgrade.

Current packages can’t be used with a nearly five-year-old release.

5000 firewall rules is 100x reasonable. Moving to a set-based approach should speed things.

You don’t need to block at the firewall those IPs to keep a device from requesting them. Returning “NXDOMAIN” from DNS means that device can’t even determine what IP belongs to ads.example.com

I still suggest upgrading and using banIP or Adblock, trying it first without your custom list.

The Atheros AR9331, @400MHz is not a powerful SoC, but should be sufficient for 100 Mbps, reasonably configured. That probably includes not keeping LuCI open when you’re not configuring things.

4 Likes

That's what I'm going to figure out. Thanks for your advice.

2 Likes

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.