banIP support thread

DBAA · September 5, 2024, 6:29pm

Suggest checking Enable Remote Logging (Enable the cgi interface to receive remote logging events.) and have the services send events to banIP. Lot more workable than trying to log individual packets.

--- from the documentation ---

CGI interface to receive remote logging events
banIP ships a basic cgi interface in '/www/cgi-bin/banip' to receive remote logging events (disabled by default). The cgi interface evaluates logging events via GET or POST request (see examples below). To enable the cgi interface set the following options:

* set 'ban_remotelog' to '1' to enbale the cgi interface
* set 'ban_remotetoken' to a secret transfer token, allowed token characters consist of '[A-Za-z]', '[0-9]', '.' and ':'

Examples to transfer remote logging events from an internal server to banIP via cgi interface:

* POST request: curl --insecure --data "<ban_remotetoken>=<suspicious IP>" https://192.168.1.1/cgi-bin/banip
* GET request: wget --no-check-certificate https://192.168.1.1/cgi-bin/banip?<ban_remotetoken>=<suspicious IP>

Please note: for security reasons use this cgi interface only internally and only encrypted via https transfer protocol.

dibdot · September 9, 2024, 7:35pm

@catsimple

DBAA:

Net Range      52.145.0.0 - 52.191.255.255
CIDR           52.145.0.0/16
               52.146.0.0/15
               52.148.0.0/14
               52.152.0.0/13
               52.160.0.0/11

banIP-1.0.0-r6 now supports multiple CIDRs in RDAP requests/replies, e.g. ...

Mon Sep  9 21:33:38 2024 user.info banIP-1.0.0-r6[7793]: suspicious IP '52.169.127.126'
Mon Sep  9 21:33:41 2024 user.info banIP-1.0.0-r6[7793]: add IP '52.169.127.126' (expiry: 2h) to blocklistv4 set
Mon Sep  9 21:33:45 2024 user.info banIP-1.0.0-r6[7793]: add IP range '52.145.0.0/16' (source: n/a, ARIN ::: expiry: 2h) to blocklistv4 set
Mon Sep  9 21:33:48 2024 user.info banIP-1.0.0-r6[7793]: add IP range '52.146.0.0/15' (source: n/a, ARIN ::: expiry: 2h) to blocklistv4 set
Mon Sep  9 21:33:51 2024 user.info banIP-1.0.0-r6[7793]: add IP range '52.148.0.0/14' (source: n/a, ARIN ::: expiry: 2h) to blocklistv4 set
Mon Sep  9 21:33:54 2024 user.info banIP-1.0.0-r6[7793]: add IP range '52.152.0.0/13' (source: n/a, ARIN ::: expiry: 2h) to blocklistv4 set
Mon Sep  9 21:33:57 2024 user.info banIP-1.0.0-r6[7793]: add IP range '52.160.0.0/11' (source: n/a, ARIN ::: expiry: 2h) to blocklistv4 set

brodrigueznu · September 9, 2024, 8:17pm

@dibdot Would it be possible to include this list in External Blocklist Feeds?

Or is there a way to include it manually?

The list is huge, so I don't know if my router is able to handle it properly.

My router is a Dynalink DL-WRX36, with 1GB of RAM.

{
        "kernel": "5.15.162",
        "hostname": "OpenWrt",
        "system": "ARMv8 Processor rev 4",
        "model": "Dynalink DL-WRX36",
        "board_name": "dynalink,dl-wrx36",
        "rootfs_type": "squashfs",
        "release": {
                "distribution": "OpenWrt",
                "version": "23.05.4",
                "revision": "r24012-d8dd03c46f",
                "target": "ipq807x/generic",
                "description": "OpenWrt 23.05.4 r24012-d8dd03c46f"
        }
}

dibdot · September 10, 2024, 3:46am

Yep, just use the Custom Feed Editor in LuCI for that.

brodrigueznu · September 10, 2024, 11:48am

I have already included it in the "Edit Custom Feeds" tab

then selected it in the "Feed Selection" tab and pressed the "Restart" button to restart the service.

However, the list does not load, it does not appear active along with the other lists.

Am I missing any steps?

JustAnotherEndUser · September 10, 2024, 2:00pm

You might want to check your 'Processing Log' for any errors that possibly point out the issue. ( such as related to downloading.) Example: I had a similar issue with some blocklists before and discovered that it was because the feeds were not downloading. Turns out the files were hosted on servers in a country being blocked by the 'CountryV4' list. I just had to make exceptions in the 'Allowlist' for domains of list providers.

Another thing that comes to mind is that there might be a formatting issue with the content of the list file download. See this post, and the response in the post below it for more information.
https://forum.openwrt.org/t/banip-support-thread/16985/2066?u=justanotherenduser

brodrigueznu · September 10, 2024, 3:12pm

I have already checked the processing log and it shows that the feed is empty.

Tue Sep 10 06:26:12 2024 user.info banIP-1.0.0-5[16242]: start banIP processing (restart)
Tue Sep 10 06:26:12 2024 user.info banIP-1.0.0-5[16242]: initialize banIP nftables namespace
Tue Sep 10 06:26:12 2024 user.info banIP-1.0.0-5[16242]: start banIP download processes
Tue Sep 10 06:26:15 2024 user.info banIP-1.0.0-5[16242]: skip empty feed 'hageziv4'
Tue Sep 10 06:26:32 2024 user.info banIP-1.0.0-5[16242]: start banIP domain lookup
Tue Sep 10 06:26:32 2024 user.info banIP-1.0.0-5[16242]: domain lookup finished in 0m 0s (allowlist, 0 domains, 0 IPs)
Tue Sep 10 06:26:32 2024 user.info banIP-1.0.0-5[16242]: domain lookup finished in 0m 0s (blocklist, 0 domains, 0 IPs)
Tue Sep 10 06:26:32 2024 user.info banIP-1.0.0-5[16242]: start detached banIP log service (/sbin/logread)

I manually downloaded the list and it contains IP addresses in plain format.

Is this list too big? It has over 240,000 entries and a size of 3.3M

What else can I check?

JustAnotherEndUser · September 10, 2024, 3:46pm

I am wondering if your 'RuleV4' is off. Specifically the part on the end.

The hagezi list looks like the same format as the Talos list. (One leading blank line, followed by one item per line. No text comments or extra spaces, etc.)

Perhaps try changing the Rulev4 to match what is set for Talos:

/^127\./{next}/^(([1-9][0-9]{0,2}\.){1}([0-9]{1,3}\.){2}(1?[0-9][0-9]?|2[0-4][0-9]|25[0-5])(\/(1?[0-9]|2?[0-9]|3?[0-2]))?)$/{printf "%s,\n",$1}

brodrigueznu · September 10, 2024, 4:35pm

With that new Rulev4 it already recognizes the list and activates it correctly but, by doing this, it deactivates the binarydefensev4 list.

Tue Sep 10 10:59:52 2024 user.info banIP-1.0.0-5[24771]: start banIP processing (restart)
Tue Sep 10 10:59:52 2024 user.info banIP-1.0.0-5[24771]: initialize banIP nftables namespace
Tue Sep 10 10:59:52 2024 user.info banIP-1.0.0-5[24771]: start banIP download processes
Tue Sep 10 11:04:10 2024 user.info banIP-1.0.0-5[24771]: skip empty feed 'binarydefensev4'
Tue Sep 10 11:05:45 2024 user.info banIP-1.0.0-5[24771]: start banIP domain lookup
Tue Sep 10 11:05:45 2024 user.info banIP-1.0.0-5[24771]: domain lookup finished in 0m 0s (allowlist, 0 domains, 0 IPs)
Tue Sep 10 11:05:45 2024 user.info banIP-1.0.0-5[24771]: domain lookup finished in 0m 0s (blocklist, 0 domains, 0 IPs)
Tue Sep 10 11:05:45 2024 user.info banIP-1.0.0-5[24771]: start detached banIP log service (/sbin/logread)

Is it possible that the IP addresses of the binarydefensev4 list are already contained within hagezi?

The problem with this is that hagezi is limited to the LAN-Forward chain, while binarydefensev4 is limited to the WAN-Input and WAN-Forward chains. So, binarydefensev4 would not be providing protection in those two chains.

EDIT: I did a comparison of both lists in an excel workbook, and found that the binarydefensev4 list is completely contained in the hagezi tif list.

That would explain why banIP determined that there was no need to process the binarydefensev4 list.

Therefore, I think the solution would be to extend the hagezi tif list limitation to the LAN-Forward, WAN-Input, and WAN-forward chains, but I still have a question about whether I should keep the limitation on ports 80 and 443.

JustAnotherEndUser · September 10, 2024, 6:13pm

There is always a potential for valid servers to be on a block list for some reason. There are many Microsoft IPs for example that are blocked for probing. Some of the lists (example, DoH) are designed with only port 80 / 443 blocks so that other services will function.
Personally, I prefer to block on all ports and zones, then just whitelist as needed. If a connection isn't working and you find something needed is being blocked, it's easy enough to turn on LAN logging in BanIP to find out what to add to the 'Allowlist' exceptions.
Also, on the subject of 'Allowlist' - if you are going to enable Geo-blocking, make sure to add exceptions to the FQDNs of all of the list providers. Several blocklist feeds are hosted in various countries across Europe, for example.

dibdot · September 10, 2024, 6:14pm

you can always disable the IP deduplication across Sets/feeds, e.g.

timur.davletshin · September 11, 2024, 2:29pm

@dibdot hi! Delete "edrop" feed from https://github.com/openwrt/packages/blob/master/net/banip/files/README.md, it is gone for quite some time. Thanks for your incredible work!

timur.davletshin · September 12, 2024, 4:46am

@dibdot another question... I ran /etc/init.d/banip reload three times and every time I got different element count. Downloaded lists didn't change between attempts but /etc/banip/banip.allowlist did. It didn't include IPv4 and ULA adresses when it was showing 1982 in element count. Is it a problem? Probably it is, because local addresses aren't included in allow list.

root@router:~# /etc/init.d/banip status
::: banIP runtime information
  + status            : active (nft: ✔, monitor: ✔)
  + version           : 1.0.0-6
  + element_count     : 1984
  + active_feeds      : allowlistv4MAC, allowlistv6MAC, allowlistv4, allowlistv6, dropv6, dropv4, myipv4, myipv6, blocklistv4MAC, blocklistv6MAC, blocklistv4, blocklistv6
  + active_devices    : wan: pppoe-wan / wan-if: wan, wan6 / vlan-allow: - / vlan-block: -
  + active_uplink     : edited_ipv4_addr/32, edited_ula_addr/128, edited_gua_addr/64
  + nft_info          : priority: -100, policy: memory, loglevel: warn, expiry: -, limit (icmp/syn/udp): 10/10/100
  + run_info          : base: /tmp, backup: /tmp/banIP-backup, report: /tmp/banIP-report
  + run_flags         : auto: ✔, proto (4/6): ✔/✔, log (pre/inp/fwd/lan): ✘/✘/✘/✘, dedup: ✔, split: ✘, custom feed: ✘, allowed only: ✘
  + last_run          : action: reload, log: logread, fetch: uclient-fetch, duration: 0m 7s, date: 2024-09-12 07:28:32
  + system_info       : cores: 2, memory: 80, device: Xiaomi Redmi Router AX6S, OpenWrt 23.05.4 r24012-d8dd03c46f
root@router:~# /etc/init.d/banip reload
root@router:~# /etc/init.d/banip status
::: banIP runtime information
  + status            : active (nft: ✔, monitor: ✔)
  + version           : 1.0.0-6
  + element_count     : 1982
  + active_feeds      : allowlistv4MAC, allowlistv6MAC, allowlistv4, allowlistv6, dropv6, dropv4, myipv4, myipv6, blocklistv4MAC, blocklistv6MAC, blocklistv4, blocklistv6
  + active_devices    : wan: pppoe-wan / wan-if: wan, wan6 / vlan-allow: - / vlan-block: -
  + active_uplink     : edited_ipv4_addr/32, edited_ula_addr/128, edited_gua_addr/64
  + nft_info          : priority: -100, policy: memory, loglevel: warn, expiry: -, limit (icmp/syn/udp): 10/10/100
  + run_info          : base: /tmp, backup: /tmp/banIP-backup, report: /tmp/banIP-report
  + run_flags         : auto: ✔, proto (4/6): ✔/✔, log (pre/inp/fwd/lan): ✘/✘/✘/✘, dedup: ✔, split: ✘, custom feed: ✘, allowed only: ✘
  + last_run          : action: reload, log: logread, fetch: uclient-fetch, duration: 0m 6s, date: 2024-09-12 07:30:03
  + system_info       : cores: 2, memory: 82, device: Xiaomi Redmi Router AX6S, OpenWrt 23.05.4 r24012-d8dd03c46f
root@router:~# /etc/init.d/banip reload
root@router:~# /etc/init.d/banip status
::: banIP runtime information
  + status            : active (nft: ✔, monitor: ✔)
  + version           : 1.0.0-6
  + element_count     : 1984
  + active_feeds      : allowlistv4MAC, allowlistv6MAC, allowlistv4, allowlistv6, dropv6, dropv4, myipv4, myipv6, blocklistv4MAC, blocklistv6MAC, blocklistv4, blocklistv6
  + active_devices    : wan: pppoe-wan / wan-if: wan, wan6 / vlan-allow: - / vlan-block: -
  + active_uplink     : edited_ipv4_addr/32, edited_ula_addr/128, edited_gua_addr/64
  + nft_info          : priority: -100, policy: memory, loglevel: warn, expiry: -, limit (icmp/syn/udp): 10/10/100
  + run_info          : base: /tmp, backup: /tmp/banIP-backup, report: /tmp/banIP-report
  + run_flags         : auto: ✔, proto (4/6): ✔/✔, log (pre/inp/fwd/lan): ✘/✘/✘/✘, dedup: ✔, split: ✘, custom feed: ✘, allowed only: ✘
  + last_run          : action: reload, log: logread, fetch: uclient-fetch, duration: 0m 7s, date: 2024-09-12 07:30:50
  + system_info       : cores: 2, memory: 81, device: Xiaomi Redmi Router AX6S, OpenWrt 23.05.4 r24012-d8dd03c46f

BTW, what's the point of backing it up if it contains ony DHCP adresses and ULA?

mercuryin · September 16, 2024, 7:26am

Hi everyone,

I have a question and was hoping to get some help. I’m trying to achieve something similar to what NetDuma does with its GeoFilter feature. Specifically, I want to block certain servers worldwide so that my Xbox doesn’t connect to them, but I want this to only affect my Xbox on the local network.

I’ve installed BanIP on OpenWRT, and I’ve added a list of worldwide servers (excluding the UK) to the Blocklist. I’ve kept the UK servers off the list, as I only want my Xbox to connect to servers in the UK. However, it doesn’t seem to be working as expected, and I’m not sure if I’m approaching this the right way.

Could you recommend how I might achieve something similar to NetDuma’s GeoFilter using BanIP? Ideally, I’d like to block the Xbox from connecting to any servers outside the UK, but I’m unsure how to configure it properly.

Any advice or tips would be much appreciated!

Thanks in advance

dibdot · September 16, 2024, 6:57pm

What does not work in detail? Anyway, please provide your banIP config as a start ... thanks.

timur.davletshin · September 19, 2024, 3:23am

Yep, banip doesn't enumerate addresses to allowlist in a correct manner. Every other time after running banip reload (dual stack, real IPv4, IPv6 GUA and ULA) two of those addresses are missing in banip.allowlist.

My config (it is mostly default with only two feeds and delay trigger incresed so it doesn't mess with adblock loading its blocklist):

config banip 'global'
	option ban_enabled '1'
	option ban_debug '0'
	option ban_autodetect '1'
	option ban_triggerdelay '30'
	option ban_fetchretry '5'
	option ban_nicelimit '0'
	option ban_filelimit '1024'
	option ban_deduplicate '1'
	option ban_nftpriority '-100'
	option ban_icmplimit '10'
	option ban_synlimit '10'
	option ban_udplimit '100'
	option ban_nftpolicy 'memory'
	option ban_blocktype 'drop'
	option ban_nftloglevel 'warn'
	option ban_logprerouting '0'
	option ban_loginput '0'
	option ban_logforwardwan '0'
	option ban_logforwardlan '0'
	option ban_loglimit '100'
	option ban_autoallowlist '1'
	option ban_autoallowuplink 'subnet'
	option ban_autoblocklist '1'
	option ban_allowlistonly '0'
	option ban_fetchcmd 'uclient-fetch'
	option ban_protov4 '1'
	option ban_protov6 '1'
	list ban_logterm 'Exit before auth from'
	list ban_logterm 'luci: failed login'
	list ban_trigger 'wan'
	list ban_ifv4 'wan'
	list ban_ifv6 'wan6'
	list ban_dev 'pppoe-wan'
	list ban_blockinput 'drop'
	list ban_blockinput 'myip'
	list ban_blockforwardwan 'drop'
	list ban_blockforwardwan 'myip'
	list ban_feed 'drop'
	list ban_feed 'myip'

dibdot · September 19, 2024, 4:19pm

Sorry, not reproducible in my setup.
Could you please PM me which IP addresses disappear between the runs and please send me the output of ...

#!/bin/sh
# save this script to /tmp/test.sh, make it executable and start it before the different banIP runs 

. "/lib/functions.sh"
. "/lib/functions/network.sh"

for iface in wan wan6; do
	network_flush_cache
	network_get_subnet uplink "${iface}"
	ban_uplink="${ban_uplink}${uplink} "
	network_get_subnet6 uplink "${iface}"
	ban_uplink="${ban_uplink}${uplink} "
done
echo "IPs: ${ban_uplink}"

Thanks!

dibdot · September 20, 2024, 8:08am

Many thanks for your support. Issue fixed in banIP 1.0.0-7:

anon77964608 · September 27, 2024, 7:31am

Opewrt 23.05.5 seems to have broken something with BanIP.
I have russia and brazil on my blocklist and ever since I've updated I can access yandex.ru and oglobo.globo.com (a russian and a brazilian websites that ive always used to check if banip was working).
Im rolling back to 23.05.4 for now.

edit: confirmed. just went back to 23.05.4 and everything is working as intended. Both of those sites are being blocked as they should.

abysso2 · September 27, 2024, 10:04am

There is no problem with incoming traffic ...

Fri Sep 27 12:00:00 2024 kern.err kernel: [91084.740370] banIP/inp-wan/drop/countryv4: IN=eth0 OUT= MAC=XXXXXXXXXXXXXXX SRC=117.199.124.238 DST=...............