banIP support thread

when i use banip,
(maybe with lots of lists, but on a x86 with enough ram and disk space)
i observe that after a couple of days
it stops working (it seems that it randomly stopped working the last time on 12. Mar while automatically updating the feeds)

::: banIP runtime information
  + status            : processing (nft: ✔, monitor: ✘)
  + version           : 0.9.4-3
  + element_count     : 0
  + active_feeds      : -
  + active_devices    : wan: eth12 / wan-if: wan, - / vlan-allow: - / vlan-block: -
  + active_uplink     : -
  + nft_info          : priority: -200, policy: performance, loglevel: warn, expiry: -
  + run_info          : base: /tmp, backup: /tmp/banIP-backup, report: /tmp/banIP-report
  + run_flags         : auto: ✔, proto (4/6): ✔/✘, log (wan-inp/wan-fwd/lan-fwd): ✔/✔/✔, dedup: ✔, split: ✘, custom feed: ✘, allowed only: ✘
  + last_run          : -
  + system_info       : cores: 64, memory: 129520, device: Thomas-Krenn.AG X11DPi-N(T), OpenWrt 23.05.0 r23497-6637af95aa

the nft table is filtering, but there is no update of the feeds etc.

if i try to restart the banip service
Failed to execute "/etc/init.d/banip restart" action: Command failed

in cli if i execute /etc/init.d/banip restart nothing happens
(i don't know how to debug this or get some more logs)

/var/run/banip.pid is empty
and the directory/var/run/banip.lock/ is empty

the only cure is to completely restart the openwrt router
and then it works for few days

/etc/config/banip:

config banip 'global'
	option ban_enabled '1'
	option ban_debug '0'
	option ban_autodetect '1'
	list ban_logterm 'Exit before auth from'
	list ban_logterm 'luci: failed login'
	list ban_logterm 'error: maximum authentication attempts exceeded'
	list ban_logterm 'sshd.*Connection closed by.*\[preauth\]'
	list ban_logterm 'SecurityEvent=\"InvalidAccountID\".*RemoteAddress='
	list ban_logterm 'received a suspicious remote IP '\''.*'\'''
	option ban_fetchinsecure '1'
	option ban_deduplicate '1'
	option ban_loginput '1'
	option ban_logforwardwan '1'
	option ban_logforwardlan '1'
	option ban_autoallowlist '1'
	option ban_autoblocklist '1'
	option ban_allowlistonly '0'
	option ban_fetchcmd 'uclient-fetch'
	option ban_protov4 '1'
	list ban_ifv4 'wan'
	list ban_dev 'eth12'
	option ban_nftpolicy 'performance'
	list ban_trigger 'wan'
	list ban_blockinput 'backscatterer'
	list ban_blockinput 'binarydefense'
	list ban_blockinput 'bogon'
	list ban_blockinput 'bruteforceblock'
	list ban_blockinput 'cinsscore'
	list ban_blockinput 'country'
	list ban_blockinput 'darklist'
	list ban_blockinput 'debl'
	list ban_blockinput 'drop'
	list ban_blockinput 'dshield'
	list ban_blockinput 'edrop'
	list ban_blockinput 'etcompromised'
	list ban_blockinput 'feodo'
	list ban_blockinput 'firehol1'
	list ban_blockinput 'firehol2'
	list ban_blockinput 'firehol3'
	list ban_blockinput 'firehol4'
	list ban_blockinput 'greensnow'
	list ban_blockinput 'ipblackhole'
	list ban_blockinput 'ipthreat'
	list ban_blockinput 'myip'
	list ban_blockinput 'nixspam'
	list ban_blockinput 'sslbl'
	list ban_blockinput 'talos'
	list ban_blockinput 'threat'
	list ban_blockinput 'threatview'
	list ban_blockinput 'tor'
	list ban_blockinput 'turris'
	list ban_blockinput 'uceprotect1'
	list ban_blockinput 'uceprotect2'
	list ban_blockinput 'uceprotect3'
	list ban_blockinput 'urlhaus'
	list ban_blockinput 'urlvir'
	list ban_blockinput 'voip'
	list ban_blockinput 'webclient'
	list ban_blockforwardwan 'backscatterer'
	list ban_blockforwardwan 'binarydefense'
	list ban_blockforwardwan 'bogon'
	list ban_blockforwardwan 'bruteforceblock'
	list ban_blockforwardwan 'cinsscore'
	list ban_blockforwardwan 'country'
	list ban_blockforwardwan 'darklist'
	list ban_blockforwardwan 'debl'
	list ban_blockforwardwan 'drop'
	list ban_blockforwardwan 'dshield'
	list ban_blockforwardwan 'edrop'
	list ban_blockforwardwan 'etcompromised'
	list ban_blockforwardwan 'feodo'
	list ban_blockforwardwan 'firehol1'
	list ban_blockforwardwan 'firehol2'
	list ban_blockforwardwan 'firehol3'
	list ban_blockforwardwan 'firehol4'
	list ban_blockforwardwan 'greensnow'
	list ban_blockforwardwan 'ipblackhole'
	list ban_blockforwardwan 'ipthreat'
	list ban_blockforwardwan 'myip'
	list ban_blockforwardwan 'nixspam'
	list ban_blockforwardwan 'proxy'
	list ban_blockforwardwan 'sslbl'
	list ban_blockforwardwan 'talos'
	list ban_blockforwardwan 'threat'
	list ban_blockforwardwan 'threatview'
	list ban_blockforwardwan 'tor'
	list ban_blockforwardwan 'turris'
	list ban_blockforwardwan 'uceprotect1'
	list ban_blockforwardwan 'uceprotect2'
	list ban_blockforwardwan 'uceprotect3'
	list ban_blockforwardwan 'urlhaus'
	list ban_blockforwardwan 'urlvir'
	list ban_blockforwardwan 'voip'
	list ban_blockforwardwan 'webclient'
	list ban_blockforwardlan 'adaway'
	list ban_blockforwardlan 'adguard'
	list ban_blockforwardlan 'adguardtrackers'
	list ban_blockforwardlan 'antipopads'
	list ban_blockforwardlan 'asn'
	list ban_blockforwardlan 'doh'
	list ban_blockforwardlan 'iblockads'
	list ban_blockforwardlan 'iblockspy'
	list ban_blockforwardlan 'oisdbig'
	list ban_blockforwardlan 'oisdnsfw'
	list ban_blockforwardlan 'oisdsmall'
	list ban_blockforwardlan 'stevenblack'
	list ban_blockforwardlan 'yoyo'
	list ban_feed 'binarydefense'
	list ban_feed 'bogon'
	list ban_feed 'bruteforceblock'
	list ban_feed 'cinsscore'
	list ban_feed 'darklist'
	list ban_feed 'debl'
	list ban_feed 'drop'
	list ban_feed 'dshield'
	list ban_feed 'edrop'
	list ban_feed 'etcompromised'
	list ban_feed 'feodo'
	list ban_feed 'firehol1'
	list ban_feed 'firehol2'
	list ban_feed 'greensnow'
	list ban_feed 'ipblackhole'
	list ban_feed 'ipthreat'
	list ban_feed 'myip'
	list ban_feed 'sslbl'
	list ban_feed 'talos'
	list ban_feed 'threat'
	list ban_feed 'threatview'
	list ban_feed 'turris'
	list ban_feed 'urlhaus'
	list ban_feed 'urlvir'
	list ban_feed 'webclient'

last report:

{
	"sets":{
		"allowlistv4MAC":{
			"cnt_elements": "0",
			"cnt_input": "",
			"input": "-",
			"cnt_forwardwan": "",
			"wan_forward": "-",
			"cnt_forwardlan": "0",
			"lan_forward": "OK",
			"port": "-"
		},
...
		"turrisv4":{
			"cnt_elements": "6282",
			"cnt_input": "0",
			"input": "OK",
			"cnt_forwardwan": "591",
			"wan_forward": "OK",
			"cnt_forwardlan": "",
			"lan_forward": "-",
			"port": "-"
		}
	},
	"timestamp": "2024-03-11 09:51:37",
	"autoadd_allow": "0",
	"autoadd_block": "5",
	"sum_sets": "32",
	"sum_setinput": "28",
	"sum_setforwardwan": "28",
	"sum_setforwardlan": "8",
	"sum_setelements": "61837",
	"sum_cntinput": "90",
	"sum_cntforwardwan": "10363",
	"sum_cntforwardlan": "212"
}

/tmp/banIP-backup/:

drwxr-xr-x    2 root     root           560 Mar 11 09:48 .
drwxrwxrwt   22 root     root           560 Mar 22 12:37 ..
-rw-r--r--    1 root     root            98 Mar 12 21:10 banIP.allowlist.gz
-rw-r--r--    1 root     root         13058 Mar 12 21:10 banIP.binarydefensev4.gz
-rw-r--r--    1 root     root          2761 Mar 12 21:10 banIP.bogonv4.gz
-rw-r--r--    1 root     root          4869 Mar 12 21:10 banIP.bruteforceblockv4.gz
-rw-r--r--    1 root     root         56289 Mar 12 21:10 banIP.cinsscorev4.gz
-rw-r--r--    1 root     root            82 Mar 12 21:10 banIP.darklistv4.gz
-rw-r--r--    1 root     root        152469 Mar 12 21:10 banIP.deblv4.gz
-rw-r--r--    1 root     root          7624 Mar 12 21:10 banIP.dropv4.gz
-rw-r--r--    1 root     root          1120 Mar 12 21:10 banIP.dshieldv4.gz
-rw-r--r--    1 root     root          2777 Mar 12 21:10 banIP.edropv4.gz
-rw-r--r--    1 root     root          1990 Mar 12 21:10 banIP.etcompromisedv4.gz
-rw-r--r--    1 root     root           548 Mar 12 21:10 banIP.feodov4.gz
-rw-r--r--    1 root     root          8520 Mar 12 21:10 banIP.firehol1v4.gz
-rw-r--r--    1 root     root        143523 Mar 12 21:10 banIP.firehol2v4.gz
-rw-r--r--    1 root     root         34477 Mar 12 21:10 banIP.greensnowv4.gz
-rw-r--r--    1 root     root         83287 Mar  7 20:10 banIP.ipblackholev4.gz
-rw-r--r--    1 root     root         30981 Mar 12 21:10 banIP.ipthreatv4.gz
-rw-r--r--    1 root     root         18036 Mar 12 21:10 banIP.myipv4.gz
-rw-r--r--    1 root     root           539 Mar 12 21:10 banIP.sslblv4.gz
-rw-r--r--    1 root     root         23477 Mar 12 21:10 banIP.talosv4.gz
-rw-r--r--    1 root     root          5768 Mar 12 21:10 banIP.threatv4.gz
-rw-r--r--    1 root     root          6925 Mar 12 21:10 banIP.threatviewv4.gz
-rw-r--r--    1 root     root         70501 Mar 12 21:10 banIP.turrisv4.gz
-rw-r--r--    1 root     root        207733 Mar 12 21:10 banIP.urlhausv4.gz
-rw-r--r--    1 root     root          1676 Mar 12 21:10 banIP.urlvirv4.gz
-rw-r--r--    1 root     root          8191 Mar 12 21:10 banIP.webclientv4.gz

Well than banIP dies in the middle of nowhere...a classic OOM condition. Limit the used cores to 4 or 8 (Set ban_cores accordingly).

1 Like

thanks, i will try

btw. i have 128gb of memory...
should be enough?

Try removing ipblackhole also in the feed. That has been down for almost a month now.

Not necessarily if banIP runs on 64 cores in parallel.

Hi. Thanks for your great work.

I have banip running on a router, configured to use the whitelist mode only. I connect with two devices to this router. Is is possible to have banip applying the rules to only one device (for example according to this device' ip address) while the other device can use the internet with banip interfering/blocking?

Yep, just add the relevant MAC/IP to your local allowlist - see the readme for details.

Hello,

First of all thanks to the developer(s) for this wonderful software.
I did read the documentation but I am still not able to achieve what I want to do.
I also want to know if what I am trying is possible with banIP.

I am trying to achieve 2 things :

  1. Block all incoming wan (input and forward) traffic from all countries, except a few (2-3)
  2. Specify a port for which the blocking should not be applied (so this should apply to all the services I host behind my router except lets say HTTPS for which I would like all the world to connect to)

For 1 I tried activating the country feed and add the countries I wanted to allow, and also set "ban_allowlistonly" to 1.
This had the consequence that I couldn't connect to anything. The "Active Feeds" on Luci was not displaying the country feed and as I don't have anything in the "Allowlist" it is kind of clear, why this is. If I unset "ban_allowlistonly", the wan input and forward are correctly dropped from these countries (after testing).
So, is it a possibility to reverse the country list and only allow the listed countries?
If not, is selecting ALL the countries except the ones I want to allow feasible? I am on x86 with plenty of RAM, but I would like to avoid putting pressure on the router if not necessary.

For 2 I don't think that it is taken into account by banIP (from what I have read in the documentation) but I still wanted to ask, maybe there is a way...

Loading ip lists for all countries in the world except select few which you want to whitelist makes no sense as this will stress the CPU a lot when processing incoming connections. I believe you can achieve whitelist blocking with BanIp but i don't know how (I'm not a user). You can also try geoip-shell which i developed for use cases exactly like this.

Just enable the allowlist only mode and put the IP segments of your preferred countries to your local allowlist or reference external IP segment files as an "External allowlist feed", e.g.

The IP segments are found here: https://www.ipdeny.com/ipblocks/data/aggregated/

Regarding your ports: Just specifiy the port ranges that you don't want - see readme for details.
Edit: Sorry, the port requirement is currently not supported in this configuration.

1 Like

Nice project! Looks promising. Will definitely have a closer look.

But wouldn't this mean that the machines on my network couldn't connect to anything except the countries I allowed?

Nope, maybe you should re-check the readme (first paragraph). Limit the feed (in this case the allowlist) to wan-input and wan-forward ...

i have limited the cores to 4

and i still get randomly banip stuck

Sorry, no idea - not reproducible on a 4-core router (Bananapi-R3) with your config.

I would also concur with @dibdot

I've got 2 x86 VMs and can't reproduce also your issue.

1 Like

hi,

How to remove those line in the log? I have many.

kern.warn kernel: [144275.270416] banIP/inp-wan/drop/countryv4: IN=wan OUT= MAC=3c:37:

Disable "Log WAN-Input" under Log Settings tab.

2 Likes

Hey @dibdot

There is a warning with gawk using the search

 /etc/init.d/banip search 35.186.224.25
gawk: cmd. line:1: warning: escape sequence `\/' treated as plain `/'
:::
::: banIP Search
:::
    Looking for IP '35.186.224.25' on 2024-04-08 06:50:34
    ---
    IP found in Set 'turrisv4'
# /etc/init.d/banip search 2600:1901:1:c36::
gawk: cmd. line:1: warning: escape sequence `\/' treated as plain `/'
gawk: cmd. line:1: warning: escape sequence `\/' treated as plain `/'
:::
::: banIP Search
:::
    Looking for IP '2600:1901:1:c36::' on 2024-04-08 06:57:10
    ---
    IP not found
1 Like

Hello

I would like to allow one specific files on some website (but not the full website): cdn.tagcommander.com/3288/tc_FFT_18.js

I tried to add it to the whitelist but it doesn't seem to work. I can only add full domain ?
thanks

Excerpt from the readme: "Furthermore, you can reference external Allowlist URLs with additional IPv4 and IPv6 feeds (see 'ban_allowurl')." ... you can't reference an external javascript file here ...