banIP support thread

Try there:

Edit maclist

Bump - is firewall4 supported. If not, i can not upgrade to 22.03-rc6 ....

If the nftables support list on github is up to date the answer is no, banIP is not yet supported. I read somewhere it's being worked on but will take time.

3 Likes

Yep, directly from didbot - banIP support thread - #709 by dibdot

Apr 12

Said that, I've just started to port/rewrite banIP to support nftables ... but it's a long way to go.

2 Likes

The old version of BanIP (0.7.10) still works in OpenWRT 22.03 - but it has become a manual process to get it to work. You have to pull legacy packages and source .ipk's from older builds - you also have to downgrade to legacy iptables. (For those who are interested).

Sorry gents, banIP is marked as broken for now - there will be no solution ready for 22.03. The migration is currently too time consuming for me. I use a CLI only prototype on my router, but it's far from being ready for prime time.

3 Likes

Uhhh, thats a pity but fair enough. Free time is precious, so i can totally understand you.

Just a question: Does that mean, that banip will be ready for 22.03 later (eg. for 22.03.2)? If so, i would stick to 20.02., because your tool increases security a lot !!!

Can you create a quick start guide or list of packages to accomplish this…it would benefit many users!

Unless you have an absolute requirement for your use case, might I suggest you simply stay with 21.02.3 and FW3.

21.02 is a stable release while 22.03.0 still only exists as a release candidate.

1 Like

how to block youtube or add youtube domain into blacklist, i tried googlevideo.com

I find very little on the progress, except for this related issue.

Where banIP.save.nft.gz is mentioned.
I'd love me some CLI only support, as there aren't many better alternatives.

Out of curiosity I checkd this.

So upgrading may be wise!

But please push whats better than the current broken state! :heart_eyes_cat:
Or some pointers about how and what needs to be done, LuCI isn't mandatory.

When 22.03 final comes out I'll look into putting together a work-around for nftables - but you're still going to have to manually do this each time you upgrade the build. The attended sys-upgrade will not work to install banip while the build is marked as broken.

Script to work with fw4/nftables. Now with ipv6!

#!/bin/sh

# IP blacklisting script for openwrt nftables

# bogons
# Emerging Threats lists 
# Blocklist.de 
URLS="https://www.team-cymru.org/Services/Bogons/bogon-bn-nonagg.txt http://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt https://www.blocklist.de/downloads/export-ips_all.txt"

# create set to contain ip addresses
if ! nft list set inet fw4 blackhole6 > /dev/null 2> /dev/null; then
  nft add set inet fw4 blackhole { flags interval\; type ipv4_addr\; auto-merge\; }
  nft add set inet fw4 blackhole6 { flags interval\; type ipv6_addr\; auto-merge\; }
fi

# insert chain to drop where source address in blackhole set
if ! nft list chain inet fw4 input_wan > /dev/null 2> /dev/null | grep @blackhole6; then
  nft insert rule inet fw4 input_wan ip saddr @blackhole drop
  nft insert rule inet fw4 input_wan ip6 saddr @blackhole6 drop
fi

# temp filename
blocklist=$(mktemp)

# add new elements to the set
for url in $URLS; do
    # download the blocklist
    curl -L -s -k "$url" 2> /dev/null
    echo
done | grep -v ^# | grep -v ^$ > "${blocklist}"

# clear old ipv4
nft flush set inet fw4 blackhole

# create ipv4 block rules
grep -v : "${blocklist}" | while read line; do
  echo add element inet fw4 blackhole { $line }
done | nft -f -

# clear old ipv6
nft flush set inet fw4 blackhole6

# create ipv6 block rules
grep : "${blocklist}" | while read line; do
  echo add element inet fw4 blackhole6 { $line }
done | nft -f -

# cleanup
rm "${blocklist}"
2 Likes

Please be so kind and open another thread for your script - thanks!

2 Likes

There is tag out, and were more wondering if the hackish steps to get it work with fw4 could be done.
May also be good for working out bugs.

Like this banip nftables file you can find in an openwrt issue, and lines to change.
Various scripts and just adding ipsets to a block list is fine and all.
But a luxury with scripts with regexp that more have looked at.
Such things are plagued with, "just scripts, so not bother sharing".
banIP is closest to a pf-badhost quivalent.

Banip is designed around iptables (which is fw3). It injects chains using the iptables command instead of nftables and the formatting is different. You would need to uninstall fw4 and install fw3 in order for it to work on 22.03. Fw3 is still available as legacy software in the repositories at this point in time. I really don't want to make a "how to" for 22.03 in it's "release candidate" state - I have a feeling that as soon as I did the final version would come out and change everything anyways... not to mention - dibdot is working on a new version for fw4 (nftables) which we are all looking forward to. He mentioned it's still a long way off in the future on a previous post.

1 Like

Second time i link to it, but here you also find.
banIP.save.nft.gz

Where I assume he has this in /etc/nftables.d/ and slightly modify the commands to work with it in the scripts.

The scripts are also quite LuCI centric, and I have less interest in the web portion of things, but this and testing is generally what is required to get a patch accepted? Have been assuming things along these lines, and not backport the firewall to support a now deprecrated LuCI functionality so to speak.

That is currently marked as broken, where it doesn't break things in any significant way currently I assume. Building with support for packages marked broken, even if it is scripts we are talking about, enables BPF in the kernel for some reason that isn't good. And banIP should be a bit "core" as a firewall.

Sorry about this! Not intending to trash post.

But thanks to awk, it takes 24 seconds on arm64 with additional lists where,

nft list set inet fw4 blackhole | wc -l

17694

Bit silly to keep the while loop for ipv4 after that processing pipe.
Everything converted to dotted quad/32, comma seperated and sent to nft that slurps.
So mot things work and populating from scratch is fast. Miss all niceties, but that bit is fine.

2 Likes

Hi, can you add an allowed_list.txt int his script?
Thanks.

This is the last post in this thread about alternative scripts.
I am tinkering a bit with it, as learn why banIP is taking time.
Quirks with how big lists nft will accept, that most need to be awk.
comm is not available in busybox, and diff is just odd.

I don't really know, or am not comfortable with nft / nftables.
And I am an awful scripter, but don't have a problem with ugly for function over form.

@powtrix you can add to the pleasure and scripty yourself, and look at pf-badhost and banIP.
Also, just change gawk to awk, as that is available in buxybox and not all may have gawk.
I'm tinkering and puzzling with awk and other thing I don't know, and would prefer removing nft flush.
And take the blackhole ipset, compare it to a new iteration and populate add and del commands to update. What you are asking is more into nft land, but by all means, hope my trash post and atrocious script additions encourage others to play with this towards banIP for fw4.

Also get a bug with at least busybox awk with the awk program added as a shell function in the script.
79.110.62.84/31 misses .85/32 (in current blocklists worked with). Where this intermittently breaks parsing existing nft ipset in order to diff --old-line-format=$'delete %l\n' --new-line-format=$'add %l\n' --unchanged-line-format='' or similar to batch process within list length that nft accept.

Tried to find something fast and sane to work around what seems to be certain short comings to make it really robust and simple without awkwards scripts. Haven't tested the awk program with gawk and others, but may or may not have hit a bug with awk in busybox, so unexpected from both program and awk that it's been an avalance of trial and error and possibly getting banned from blocklist.de :stuck_out_tongue:

Maybe another nft issue, and not awk, but what I wanted ended up in a ditch.
And that ripping ip's in largely the same way from the nft set, diffing and doing adding and deleting fast without flushing. Most likely that bothersome ip-range syntax you find in nft list output, because there are actually two problems, and same problem with awk and gawk.

This is not UNIX friendly.

So another nft bug most likely. So my deppest sympathies for banIP development.

nft list set inet fw4 blackhole | grep 85.203.22

  	    85.203.22.76/31, 85.203.22.180,

nft delete element inet fw4 blackhole { 85.203.22.76/32 }

Error: interval not found in set
delete element inet fw4 blackhole { 85.203.22.76/32 }
^^^^^^^^^^^^^^^
Error: Could not process rule: No such file or directory
delete element inet fw4 blackhole { 85.203.22.76/32 }
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

This and the "interval" mannerism, ie. ip-ip, makes this awful and makes you flush.
I was batching in 1000 at a time, less for deletion to work this out.
Arbitrarily being hit by this and ip ranges/intervals, and nft as a tool is not intelligent enough for the operation listed, while grouping without being told and fed /32 excluusively.
So if 1000 ip operations fail, check and fix what rejected it or write c and patch nft?

FYI, less than two minutes for 36 000 ip's, and subsequent updates and chances less than 30 seconds. Starting with 1,4MB uncompressed text of raw input for various sources with overlaps.

For basic useable, ip's should be able to be removed from sets with nft from command line the same way they was entered. Having lots of ipsets doesn't remove the problem, I don't understand why biection and mapping through this grouping logic -- and two of them -- isn there.