banIP: new project needs testers & feedback


#1

Hi,

let me introduce my latest project called "banIP" - a package to block incoming & outgoing ip adresses/subnets via ipset. Screenshots will follow in the second post.

Features:

  • a shell script which uses ipset and iptables to ban a large number of IP addresses published in IP blacklists
  • support blocking by ASN numbers
  • support blocking by iso country codes
  • support local white & blacklist (IPv4, IPv6 & CIDR notation)
  • auto-add unsuccessful ssh login attempts to local blacklist
  • auto-add the uplink subnet to local whitelist
  • per source configuration of SRC (incoming) and DST (outgoing)
  • supports IPv4 & IPv6 (the latter one is completely untested!)
  • Strong LuCI support:
    ** easy interface to change all aspects of your iptables/ipset configuration on the fly
    ** integrated IPSet-Lookup
    ** integrated RIPE-Lookup
    ** Log-Viewer & online configuration of white- & blacklist
    [... maybe more ...] :wink:

Downloads: This is really an early bird, even if it works for me "in production", expect bugs, unfinished things etc. Both packages are now part of the official package repo, download links:

Please note: This project is only intended for current snapshot release level or branch 18.x!

It would be nice to get feedback, in particular regarding IPv6 - I have no experience with this and no chance to test it on my own ... any feedback/questions etc. are welcome! :wink:

Have fun!
Dirk


Blocking Facebook
Adblock support thread
Geo-Filtering IP Addresses by Country Code
How to setup a firewall "white-list" and how to use ip-set on LEDE?
Optimized build for the D-Link DIR-860L
#2

Screenshots from LuCI interface:

  • Overview page with Runtime Infomation (will be updated dynamically via XHR poll)
    banIP_01

  • Currently supported IP blacklists
    banIP_02

  • Extra options (you can change firewall aspects via "Addional Field" select box
    banIP_03

  • IPSet-Lookup for all active IPsets
    banIP_04

  • RIPE-Lookup with many pre-configured reports - output is raw json
    banIP_05

  • Advanced section overview
    banIP_06


#4

Can you precise that "current"? Will this be true even in 2 years time from now?
I'm coming from the OpenWrt wiki, where many "current" or "currently" are heavily outdated, years after they have been added and never got removed again...


#5

Notably the LuCI frontend part requires the latest changes in master, which are not in 17.x or 18.x branch.


#6

I've been using firehol's ipset lists for some time, finally someone made a frontend for that. Unfortunately there's no IPv6 here. D:

I'll add your packages to my next build. Thank you.


#7

@dibdot finally a project like sub2rbl but with a LuCI frontend for OpenWRT. Quick question, how is the ram usage with ipsets?
Also, tiny nitpick, it is ransomware, not ransomeware.
Thank you!


#8

On the netfilter mailing list I found this calculation formula: https://www.spinics.net/lists/netfilter/msg56265.html

banIP always tries to start with a very small hashsize. Anyway, the kernel rounds this up to the first (lowest) valid hashsize during ipset creation. Compared to dns level blocking the RAM usage is very low.


#9

@dibdot thanks for this, looks really useful.

I installed and all seems to work except blocking by iso country codes, anything I can do to tweak/test?

Also, I seem my subnet has been auto-added to white list, should it be there? Or does that disable banip on my network?


#10

Please provide debug logs - thanks!

That's the intended behaviour to keep your uplink/subnet always accessible.

IPv6 testing/feedback would be fine! :wink:


#11

Did some more testing; "auto-add unsuccessful ssh login attempts to local blacklist" this feature seems to be non-functional as well for me.

Regarding "except blocking by iso country codes" in situations where multiple countries are added, the most recently added country, seems to be the only one blocked.

Wed Aug  1 22:20:25 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: country, mode: create, settype: net, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 8711/0/8711, time(s): 12
Wed Aug  1 22:20:25 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: country_6, mode: create, settype: net, setipv: inet6, ruletype: src+dst, count(sum/ip/cidr): 1771/0/1771, time(s): 12
Wed Aug  1 22:20:25 2018 user.info banIP-[0.0.1]: 18 IPSets with overall 165102 IPs/Prefixes loaded successfully (Linksys WRT1900ACS, Cantenna_22-06-18 v.1.15- Lede SNAPSHOT r6865+1-419238f)
Wed Aug  1 22:20:25 2018 user.debug banIP-[0.0.1]: f_jsnup ::: status: enabled, setcnt: 18, cnt: 165102
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_jsnup ::: status: running, setcnt: 0, cnt: 0
Wed Aug  1 22:20:46 2018 user.info banIP-[0.0.1]: start banIP processing (start)
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_main  ::: fetch_util: /usr/bin/curl (built-in), iface: lan, dev: br-lan, mem_total: 511, mem_free: 336, max_queue: 32
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: -, mode: initial, chain: banIP, ruleset: input_wan_rule forwarding_wan_rule input_lan_rule forwarding_lan_rule, ruleset_6: input_wan_rule forwarding_wan_rule input_lan_rule forwarding_lan_rule
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_main  ::: name: whitelist, src_on: 1
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_main  ::: name: whitelist_6, src_on: 0
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: whitelist_6, mode: flush
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_main  ::: name: blacklist, src_on: 1
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_main  ::: name: blacklist_6, src_on: 0
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: blacklist_6, mode: flush
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_main  ::: name: tor, src_on: 1
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_main  ::: name: threat, src_on: 1
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_main  ::: name: debl, src_on: 1
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_main  ::: name: debl_6, src_on: 0
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: debl_6, mode: flush
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: blacklist, mode: create, settype: net, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 0/0/0, time(s): 0
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_main  ::: name: myip, src_on: 1
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_main  ::: name: myip_6, src_on: 0
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: myip_6, mode: flush
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_main  ::: name: bogon, src_on: 1
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_main  ::: name: bogon_6, src_on: 1
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_main  ::: name: yoyo, src_on: 1
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: zeus, src_on: 1
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: sslbl, src_on: 1
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: ransomeware, src_on: 1
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: feodo, src_on: 1
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: dshield, src_on: 1
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: proxy, src_on: 1
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: iblocklist, src_on: 1
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: drop, src_on: 1
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: drop_6, src_on: 0
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: drop_6, mode: flush
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: edrop, src_on: 1
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: firehol1, src_on: 0
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: whitelist, mode: create, settype: net, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 1/0/1, time(s): 1
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: firehol1, mode: flush
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: firehol2, src_on: 0
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: firehol2, mode: flush
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: firehol3, src_on: 0
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: firehol3, mode: flush
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: firehol4, src_on: 0
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: firehol4, mode: flush
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: country, src_on: 1
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: country_6, src_on: 1
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: asn, src_on: 0
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: asn, mode: flush
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: asn_6, src_on: 0
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: asn_6, mode: flush
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: iblocklist, mode: create, settype: net, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 0/0/0, time(s): 0
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: edrop, mode: create, settype: net, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 112/0/112, time(s): 0
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: dshield, mode: create, settype: net, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 20/0/20, time(s): 0
Wed Aug  1 22:20:48 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: threat, mode: create, settype: net, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 2412/1567/845, time(s): 2
Wed Aug  1 22:20:50 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: drop, mode: create, settype: net, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 825/0/825, time(s): 3
Wed Aug  1 22:20:50 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: ransomeware, mode: create, settype: ip, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 304/304/0, time(s): 3
Wed Aug  1 22:20:50 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: zeus, mode: create, settype: ip, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 108/108/0, time(s): 3
Wed Aug  1 22:20:50 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: feodo, mode: create, settype: ip, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 1464/1464/0, time(s): 3
Wed Aug  1 22:20:50 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: proxy, mode: create, settype: ip, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 2749/2749/0, time(s): 3
Wed Aug  1 22:20:51 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: yoyo, mode: create, settype: ip, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 11843/11843/0, time(s): 5
Wed Aug  1 22:20:51 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: sslbl, mode: create, settype: ip, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 99/99/0, time(s): 4
Wed Aug  1 22:20:51 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: tor, mode: create, settype: ip, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 920/920/0, time(s): 5
Wed Aug  1 22:20:52 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: bogon, mode: create, settype: net, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 3239/0/3239, time(s): 6
Wed Aug  1 22:20:52 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: myip, mode: create, settype: ip, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 3692/3692/0, time(s): 6
Wed Aug  1 22:20:52 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: debl, mode: create, settype: ip, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 27287/27287/0, time(s): 6
Wed Aug  1 22:20:56 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: country_6, mode: create, settype: net, setipv: inet6, ruletype: src+dst, count(sum/ip/cidr): 1771/0/1771, time(s): 9
Wed Aug  1 22:20:58 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: country, mode: create, settype: net, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 8711/0/8711, time(s): 11
Wed Aug  1 22:20:59 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: bogon_6, mode: create, settype: net, setipv: inet6, ruletype: src+dst, count(sum/ip/cidr): 99545/0/99545, time(s): 13
Wed Aug  1 22:20:59 2018 user.info banIP-[0.0.1]: 18 IPSets with overall 165102 IPs/Prefixes loaded successfully (Linksys WRT1900ACS, Cantenna_22-06-18 v.1.15- Lede SNAPSHOT r6865+1-419238f)
Wed Aug  1 22:20:59 2018 user.debug banIP-[0.0.1]: f_jsnup ::: status: enabled, setcnt: 18, cnt: 165102

#12

Thanks for testing! :wink:
What's your testcase? Currently only lines like below will be auto-added (whenever you've reached the max-attempts):

Exit before auth (user 'root', 3 fails): Max auth tries reached - user 'root' from 10.168.1.103:53372

Regarding iso codes: which country codes did you use for testing?

Thanks again!


#13

confirmed & fixed with the next update (same bug applied to dynamic ASN IPsets, too) - thanks!


#14

(sorry for late reply, was asleep)

I use ssh key+pass dropbear.

Country codes DE, RU, CN

But I see you made progress anyways :wink: Looking forward to you next update.


#15

Could you please send me a logfile/logread excerpt from the intentionally failed login via PM?

Thanks!


#16

@cantenna I've reproduced & fixed that issue, too. I need no further logs ... thanks.


#17

banIP v 0.0.2 is now in my google drive folder (see first post), with the following changes:

  • fix auto-add function of failed ssh logins
  • fix dynamic ASN & Country IPSet creation where multiple sources are selected
  • fix "ransomware" typo
  • updated LuCI components - should work with 18.06 release & latest snapshots

Please remove the old version before you update and reset the LuCI cache (rm -rf /tmp/luci-*)

Happy testing! :wink:


#18

Literally just working on getting that log to you now, came here to double check what you were asking for again. Awesome that you got it done anyways. Will give the update a go, thanks again:)


#19

Thanks for the update. Country block does seem to work but for some reason, occasionally after a reboot, country ipset fails to be known to banip i.e.) Country is missing from IPSet-Lookup. EDIT: increasing/adding trigger delay of 10 seconds seems to have helped

Haven't tested ssh block yet, will do next.

Is it possible to o stuff via console? I know it's early days.


#20

Check the debug log regarding failed downloads ... usually you can fine tune this with a reduced number of parallel processes (you've raised this to 32 if I remember right :wink: ) plus a higher trigger delay (default: 2).

/etc/init.d/banip start

to refresh your IPSets ... or did you mean something different!?


#21

That's great. Wasn't sure if restart would be same as refresh, thanks again, really useful package for LEDE