banIP support thread

Community Builds, Projects & Packages
1

Hi,

in OpenWrt snapshot package repo you'll find the banIP package:

latest snapshot version: banIP 1.5.5-2 plus luci companion package

stable OpenWrt version 24.10.x.: banIP 1.5.5-2 plus luci companion package

Link to the latest banIP documentation

Feel free to test, ask questions or make suggestions.

Changelog

---
** update 1.5.5-2**

  • fixed a JSON reporting issue (when the map and NFT counters are disabled)
  • optimized the getfetch function call within the reporting function
  • removed the stale IPv6 links in the becyber feed
  • cosmetics
    ---
    ** release 1.5.5-1**
  • added a geoIP Map to show home IPs and potential attacker IPs on a leafletjs based map
  • significantly improved the reporting performance on multicore hardware
  • removed aria2 support (it doesn't support post data requests)
  • removed the following outbound feeds due to too many false positives:
    adaway, adguard, adguardtrackers, antipopads, oisdbig, oisdnsfw, oisdsmall, stevenblack and yoyo
  • renamed the banIP command "survey" to "content"
  • various other small tweaks
  • update the readme
    ---
    ** update 1.5.3-3**
  • fix a race condition in the process scheduler
  • sync the banIP country file with ipdeny feed
  • refine etag handling with country/asn feeds
  • refine logging with country/asn feeds
  • refine the banIP status output (incl. LuCI changes)
    ---
    ** update 1.5.3-2**
  • fix uci config processing (ban_logterm)
  • update the readme
    ---
    ** release 1.5.3-1**
  • optimized uci config processing (list options)
  • optimized icmp rules in pre-routing (thanks @brada)
  • set inbound marker in pre-routing only if inbound logging is enabled (fixes #26044)
  • fix cornercase in Set removal function
  • print chain-, set- and rules-counter in the banIP status
  • clean up logging und download queue handling
  • update the readme
    ---
    ** release 1.5.2-1**
  • add memory measurements:
  • free memory in MB (MemAvailable from /proc/meminfo)
  • script run max. used RAM in MB (VmHWM from /proc/$$/status)
  • removed the obsolete (domain) lookup command in init script
  • update the readme
  • LuCI update
    ---
    ** release 1.5.1-1**
  • optimized the f_nftload function
  • reduced the prerouting priority to -175
  • optimized the output of the f_survey function
  • removed a needless fw4 call/check
  • no longer skips regular blocklist feeds in "allowlist only" mode
  • optimized init checks
  • turris feed: enable IPv6 parsing, too (provided by @curbengh)
  • update the readme
    ---
    ** update 1.5.0-6**
  • fixed the incomplete rule maintainance during banIP reloads
  • fixed the Set query function (if the Set counters are disabled)
    ---
    ** update 1.5.0-5**
  • fix a reporting issue with multiple ports/protocols
    ---
    ** update 1.5.0-4**
  • Block HTTP/3 by including UDP in the feeds that have the 80 443 ports in their flag
    ---
    ** update 1.5.0-3**
  • better logging of nft load errors
  • update the readme
    ---
    ** update 1.5.0-2**
  • fix typo in the log_outbound target
  • update the readme
    ---
    ** release 1.5.0-1**
  • change the chain structure: only two regular chains contain the generated banIP sets.
    “_inbound” covers the base chains WAN-Input and WAN-Forward, ‘_outbound’ covers the base chain LAN-Forward.
  • pre-configure the default chains for every feed in the banip.feeds json file, no longer blocks
    selected feeds in all chains by default
  • it's now possible to split country and asn Sets by country or asn (disabled by default)
  • support Set counters to report easily suspicious IPs per Set (disabled by default)
  • make it possible, to opt out certain chains from the deduplication process
  • the element search now returns all matches (and not only the first one)
  • the report engine now includes statistics about the Inbound & Outbound chains and the Set counters (optional)
  • save the temp. files of possible nft loading errors in "/tmp/banIP-errors" by default for easier debugging
  • various code improvements
  • remove ssbl feed (deprecated)
  • add two new vpn feeds
  • update the readme

Have fun!
Dirk

48 Likes
2

a new LuCI frontend is currently under construction ... :wink:

7 Likes
4

Can you precise that "current"? Will this be true even in 2 years time from now?
I'm coming from the OpenWrt wiki, where many "current" or "currently" are heavily outdated, years after they have been added and never got removed again...

1 Like
5

Notably the LuCI frontend part requires the latest changes in master, which are not in 17.x or 18.x branch.

1 Like
6

I've been using firehol's ipset lists for some time, finally someone made a frontend for that. Unfortunately there's no IPv6 here. D:

I'll add your packages to my next build. Thank you.

1 Like
7

@dibdot finally a project like sub2rbl but with a LuCI frontend for OpenWRT. Quick question, how is the ram usage with ipsets?
Also, tiny nitpick, it is ransomware, not ransomeware.
Thank you!

1 Like
8

On the netfilter mailing list I found this calculation formula: https://www.spinics.net/lists/netfilter/msg56265.html

banIP always tries to start with a very small hashsize. Anyway, the kernel rounds this up to the first (lowest) valid hashsize during ipset creation. Compared to dns level blocking the RAM usage is very low.

3 Likes
9

@dibdot thanks for this, looks really useful.

I installed and all seems to work except blocking by iso country codes, anything I can do to tweak/test?

Also, I seem my subnet has been auto-added to white list, should it be there? Or does that disable banip on my network?

1 Like
10

Please provide debug logs - thanks!

That's the intended behaviour to keep your uplink/subnet always accessible.

IPv6 testing/feedback would be fine! :wink:

2 Likes
11

Did some more testing; "auto-add unsuccessful ssh login attempts to local blacklist" this feature seems to be non-functional as well for me.

Regarding "except blocking by iso country codes" in situations where multiple countries are added, the most recently added country, seems to be the only one blocked.

Wed Aug  1 22:20:25 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: country, mode: create, settype: net, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 8711/0/8711, time(s): 12
Wed Aug  1 22:20:25 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: country_6, mode: create, settype: net, setipv: inet6, ruletype: src+dst, count(sum/ip/cidr): 1771/0/1771, time(s): 12
Wed Aug  1 22:20:25 2018 user.info banIP-[0.0.1]: 18 IPSets with overall 165102 IPs/Prefixes loaded successfully (Linksys WRT1900ACS, Cantenna_22-06-18 v.1.15- Lede SNAPSHOT r6865+1-419238f)
Wed Aug  1 22:20:25 2018 user.debug banIP-[0.0.1]: f_jsnup ::: status: enabled, setcnt: 18, cnt: 165102
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_jsnup ::: status: running, setcnt: 0, cnt: 0
Wed Aug  1 22:20:46 2018 user.info banIP-[0.0.1]: start banIP processing (start)
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_main  ::: fetch_util: /usr/bin/curl (built-in), iface: lan, dev: br-lan, mem_total: 511, mem_free: 336, max_queue: 32
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: -, mode: initial, chain: banIP, ruleset: input_wan_rule forwarding_wan_rule input_lan_rule forwarding_lan_rule, ruleset_6: input_wan_rule forwarding_wan_rule input_lan_rule forwarding_lan_rule
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_main  ::: name: whitelist, src_on: 1
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_main  ::: name: whitelist_6, src_on: 0
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: whitelist_6, mode: flush
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_main  ::: name: blacklist, src_on: 1
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_main  ::: name: blacklist_6, src_on: 0
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: blacklist_6, mode: flush
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_main  ::: name: tor, src_on: 1
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_main  ::: name: threat, src_on: 1
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_main  ::: name: debl, src_on: 1
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_main  ::: name: debl_6, src_on: 0
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: debl_6, mode: flush
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: blacklist, mode: create, settype: net, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 0/0/0, time(s): 0
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_main  ::: name: myip, src_on: 1
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_main  ::: name: myip_6, src_on: 0
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: myip_6, mode: flush
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_main  ::: name: bogon, src_on: 1
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_main  ::: name: bogon_6, src_on: 1
Wed Aug  1 22:20:46 2018 user.debug banIP-[0.0.1]: f_main  ::: name: yoyo, src_on: 1
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: zeus, src_on: 1
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: sslbl, src_on: 1
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: ransomeware, src_on: 1
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: feodo, src_on: 1
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: dshield, src_on: 1
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: proxy, src_on: 1
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: iblocklist, src_on: 1
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: drop, src_on: 1
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: drop_6, src_on: 0
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: drop_6, mode: flush
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: edrop, src_on: 1
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: firehol1, src_on: 0
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: whitelist, mode: create, settype: net, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 1/0/1, time(s): 1
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: firehol1, mode: flush
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: firehol2, src_on: 0
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: firehol2, mode: flush
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: firehol3, src_on: 0
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: firehol3, mode: flush
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: firehol4, src_on: 0
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: firehol4, mode: flush
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: country, src_on: 1
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: country_6, src_on: 1
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: asn, src_on: 0
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: asn, mode: flush
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_main  ::: name: asn_6, src_on: 0
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: asn_6, mode: flush
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: iblocklist, mode: create, settype: net, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 0/0/0, time(s): 0
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: edrop, mode: create, settype: net, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 112/0/112, time(s): 0
Wed Aug  1 22:20:47 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: dshield, mode: create, settype: net, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 20/0/20, time(s): 0
Wed Aug  1 22:20:48 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: threat, mode: create, settype: net, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 2412/1567/845, time(s): 2
Wed Aug  1 22:20:50 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: drop, mode: create, settype: net, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 825/0/825, time(s): 3
Wed Aug  1 22:20:50 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: ransomeware, mode: create, settype: ip, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 304/304/0, time(s): 3
Wed Aug  1 22:20:50 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: zeus, mode: create, settype: ip, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 108/108/0, time(s): 3
Wed Aug  1 22:20:50 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: feodo, mode: create, settype: ip, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 1464/1464/0, time(s): 3
Wed Aug  1 22:20:50 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: proxy, mode: create, settype: ip, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 2749/2749/0, time(s): 3
Wed Aug  1 22:20:51 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: yoyo, mode: create, settype: ip, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 11843/11843/0, time(s): 5
Wed Aug  1 22:20:51 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: sslbl, mode: create, settype: ip, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 99/99/0, time(s): 4
Wed Aug  1 22:20:51 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: tor, mode: create, settype: ip, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 920/920/0, time(s): 5
Wed Aug  1 22:20:52 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: bogon, mode: create, settype: net, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 3239/0/3239, time(s): 6
Wed Aug  1 22:20:52 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: myip, mode: create, settype: ip, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 3692/3692/0, time(s): 6
Wed Aug  1 22:20:52 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: debl, mode: create, settype: ip, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 27287/27287/0, time(s): 6
Wed Aug  1 22:20:56 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: country_6, mode: create, settype: net, setipv: inet6, ruletype: src+dst, count(sum/ip/cidr): 1771/0/1771, time(s): 9
Wed Aug  1 22:20:58 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: country, mode: create, settype: net, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 8711/0/8711, time(s): 11
Wed Aug  1 22:20:59 2018 user.debug banIP-[0.0.1]: f_ipset ::: name: bogon_6, mode: create, settype: net, setipv: inet6, ruletype: src+dst, count(sum/ip/cidr): 99545/0/99545, time(s): 13
Wed Aug  1 22:20:59 2018 user.info banIP-[0.0.1]: 18 IPSets with overall 165102 IPs/Prefixes loaded successfully (Linksys WRT1900ACS, Cantenna_22-06-18 v.1.15- Lede SNAPSHOT r6865+1-419238f)
Wed Aug  1 22:20:59 2018 user.debug banIP-[0.0.1]: f_jsnup ::: status: enabled, setcnt: 18, cnt: 165102
2 Likes
12

Thanks for testing! :wink:
What's your testcase? Currently only lines like below will be auto-added (whenever you've reached the max-attempts):

Exit before auth (user 'root', 3 fails): Max auth tries reached - user 'root' from 10.168.1.103:53372

Regarding iso codes: which country codes did you use for testing?

Thanks again!

1 Like
13

confirmed & fixed with the next update (same bug applied to dynamic ASN IPsets, too) - thanks!

2 Likes
14

(sorry for late reply, was asleep)

I use ssh key+pass dropbear.

Country codes DE, RU, CN

But I see you made progress anyways :wink: Looking forward to you next update.

1 Like
15

Could you please send me a logfile/logread excerpt from the intentionally failed login via PM?

Thanks!

1 Like
16

@cantenna I've reproduced & fixed that issue, too. I need no further logs ... thanks.

1 Like
17

banIP v 0.0.2 is now in my google drive folder (see first post), with the following changes:

  • fix auto-add function of failed ssh logins
  • fix dynamic ASN & Country IPSet creation where multiple sources are selected
  • fix "ransomware" typo
  • updated LuCI components - should work with 18.06 release & latest snapshots

Please remove the old version before you update and reset the LuCI cache (rm -rf /tmp/luci-*)

Happy testing! :wink:

5 Likes
18

Literally just working on getting that log to you now, came here to double check what you were asking for again. Awesome that you got it done anyways. Will give the update a go, thanks again:)

1 Like
19

Thanks for the update. Country block does seem to work but for some reason, occasionally after a reboot, country ipset fails to be known to banip i.e.) Country is missing from IPSet-Lookup. EDIT: increasing/adding trigger delay of 10 seconds seems to have helped

Haven't tested ssh block yet, will do next.

Is it possible to o stuff via console? I know it's early days.

1 Like
20

Check the debug log regarding failed downloads ... usually you can fine tune this with a reduced number of parallel processes (you've raised this to 32 if I remember right :wink: ) plus a higher trigger delay (default: 2).

/etc/init.d/banip start

to refresh your IPSets ... or did you mean something different!?

1 Like
21

That's great. Wasn't sure if restart would be same as refresh, thanks again, really useful package for LEDE