banIP support thread

Although this Serverfault question is about IPv6 in from-torange/P2P-style format, it does seem that IPSet supports adding IP's for IPv4 in the from-torange/P2P-style format nowadays.

Ok, so I've been running banip with a HE.net 6in4 tunnel (@hisham2630 I looked again, and HE now has a local tunnel endpoint with acceptable latency). So far, a summary of bits which have arisen:

  1. Whitelist functionality does not seem to cover IPv6, it seems there is no creation of IPv6 whitelist sets, despite my whitelist having an IPv6 address in it.
  2. IPv6 sets are applied to WAN interfaces having only IPv4 connectivity, i.e. the interfaces are added to the IPv6 banIP list, and vice versa for interfaces having only IPv4 address.

For 2 in the above list, I have tried disabling the "builtin IPv6 functionality" for the interfaces. I have also tried having a separate firewall zone for IPv4 with it restricted to only IPv4, and the same for an IPv6 zone, also limited to only IPv6.

Thank you for all of your hard work on this! Please contact me if I can provide any more troubleshooting info, it might be a bit delayed this time as I am back at work, and rather busy all around. :frowning:

1 Like

Hi, many many thanks for your testing efforts & your support! :+1:
It would be nice to get a full (debug enabled) banIP runtime log via PM or email to my maintainers address - and take your time ,,, currently my daytime job is very tempting, too. :wink:

Edit: Please provide your whitelist (with IPv6 addresses too, cause I can't reproduce that).

Will do so, my next chance to strip down to a "barebones" config/firmware will likely only be this coming weekend. (One day I'll have the luxury of a spare router for to mess about with, without risking unemployment as a side effect. :stuck_out_tongue: )

I'll send an archive of everything related, and likely some stuff which is not.

I am a complete beginner with this.
Not into programming etc etc.

I loaded BanIP onto the latest WRT3200acm OpenWRT firmware (which is great).
I previously used PiHole, which is MUCH more user friendly, though nowhere near as elegant a solution as BanIP (and Adblock for OpenWRT previously). They seem to crash each other so it's one or the other it seems to me.

(I want simple adblocking, google, facebook, eBay advertising etc. I am fed up at the junk that is downloaded visiting websites).

Despite loading most of the modules, I still get adverts, eg googleads, doubleclick, etc.
Why?

Can I wildcard sites?
How do I know which sites are blocked that I don't want to be?
Why can't I load a list from many of the websites that advertise blacklists, eg Filterlists.

The difficult functionality of this app and OpenWRT, generally, prevents Joe Public from taking up these programmes en masse.

PS I'm not a simpleton, and I don't want to learn computer programming like an expert, but a little tinkering is OK, and good simple to follow help pages would be a bonus.

Then you are on the wrong thread. What you need is adblock.

1 Like

definitely not, with me both programs always run in parallel ...:wink:

then use adblock, not banIP. The main purpose of the latter one is blocking of incoming ip addresses or subnets. Of course, you could also block outgoing ips but it's less effective than adblock.

What about updating the block lists for something more useful overall, firehol_level1 supersedes a bunch of lists, so maybe we'd have room for turris_greylist, iblocklist_pedos and so on. Just food for thought.

Could you provide a working config with your enhancements / other sources? Feel free to provide it here in the forum or publish a pull request at the package repo.

Thanks!

First of all, thank you @dibdot for this awesome addition to OpenWRT. Very nice work and great efforts!

I've read through this thread (and searched) for answer to this question, but I have not found a definitive answer yet. How often should banIP processing occur?

I saw mention of a regular blocklist update via cron at 6am daily. But I do not see /etc/crontabs/root on my router. I can create it if that's the next step, but was curious if it was supposed to be created as part of the banIP package install.

The main reason I ask is that I am seeing 'start banIP processing (start)' messages in my system log every couple minutes. Below is just a snippet of the frequency at which I see the messages. This has been occurring at this interval since installing banIP. Is this expected or should I only see a 'start banIP processing' once daily (if the cronjob was in place)?

Thu May  9 10:25:50 2019 user.info banIP-[0.1.0]: start banIP processing (start)
Thu May  9 10:26:41 2019 user.info banIP-[0.1.0]: 21 IPSets with overall 172781 IPs/Prefixes loaded successfully (UBNT-ERX, OpenWrt SNAPSHOT r9945-bc85640cdc)
Thu May  9 10:28:37 2019 user.info banIP-[0.1.0]: start banIP processing (start)
Thu May  9 10:29:30 2019 user.info banIP-[0.1.0]: 21 IPSets with overall 172781 IPs/Prefixes loaded successfully (UBNT-ERX, OpenWrt SNAPSHOT r9945-bc85640cdc)
Thu May  9 10:30:54 2019 user.info banIP-[0.1.0]: start banIP processing (start)
Thu May  9 10:31:45 2019 user.info banIP-[0.1.0]: 21 IPSets with overall 172780 IPs/Prefixes loaded successfully (UBNT-ERX, OpenWrt SNAPSHOT r9945-bc85640cdc)
Thu May  9 10:32:41 2019 user.info banIP-[0.1.0]: start banIP processing (start)
Thu May  9 10:33:34 2019 user.info banIP-[0.1.0]: 21 IPSets with overall 172780 IPs/Prefixes loaded successfully (UBNT-ERX, OpenWrt SNAPSHOT r9945-bc85640cdc)
Thu May  9 10:35:13 2019 user.info banIP-[0.1.0]: start banIP processing (start)
Thu May  9 10:36:05 2019 user.info banIP-[0.1.0]: 21 IPSets with overall 172779 IPs/Prefixes loaded successfully (UBNT-ERX, OpenWrt SNAPSHOT r9945-bc85640cdc)
Thu May  9 10:39:32 2019 user.info banIP-[0.1.0]: start banIP processing (start)
Thu May  9 10:40:23 2019 user.info banIP-[0.1.0]: 21 IPSets with overall 172782 IPs/Prefixes loaded successfully (UBNT-ERX, OpenWrt SNAPSHOT r9945-bc85640cdc)
Thu May  9 10:41:49 2019 user.info banIP-[0.1.0]: start banIP processing (start)
Thu May  9 10:42:40 2019 user.info banIP-[0.1.0]: 21 IPSets with overall 172782 IPs/Prefixes loaded successfully (UBNT-ERX, OpenWrt SNAPSHOT r9945-bc85640cdc)
Thu May  9 10:45:38 2019 user.info banIP-[0.1.0]: start banIP processing (start)
Thu May  9 10:46:28 2019 user.info banIP-[0.1.0]: 21 IPSets with overall 172782 IPs/Prefixes loaded successfully (UBNT-ERX, OpenWrt SNAPSHOT r9945-bc85640cdc)
Thu May  9 10:49:57 2019 user.info banIP-[0.1.0]: start banIP processing (start)
Thu May  9 10:50:49 2019 user.info banIP-[0.1.0]: 21 IPSets with overall 172720 IPs/Prefixes loaded successfully (UBNT-ERX, OpenWrt SNAPSHOT r9945-bc85640cdc)
Thu May  9 10:52:30 2019 user.info banIP-[0.1.0]: start banIP processing (start)
Thu May  9 10:53:21 2019 user.info banIP-[0.1.0]: 21 IPSets with overall 172720 IPs/Prefixes loaded successfully (UBNT-ERX, OpenWrt SNAPSHOT r9945-bc85640cdc)
Thu May  9 10:55:02 2019 user.info banIP-[0.1.0]: start banIP processing (start)
Thu May  9 10:55:54 2019 user.info banIP-[0.1.0]: 21 IPSets with overall 172721 IPs/Prefixes loaded successfully (UBNT-ERX, OpenWrt SNAPSHOT r9945-bc85640cdc)
Thu May  9 10:57:19 2019 user.info banIP-[0.1.0]: start banIP processing (start)
Thu May  9 10:58:10 2019 user.info banIP-[0.1.0]: 21 IPSets with overall 172721 IPs/Prefixes loaded successfully (UBNT-ERX, OpenWrt SNAPSHOT r9945-bc85640cdc)
Thu May  9 10:59:51 2019 user.info banIP-[0.1.0]: start banIP processing (start)
Thu May  9 11:00:39 2019 user.info banIP-[0.1.0]: 21 IPSets with overall 172722 IPs/Prefixes loaded successfully (UBNT-ERX, OpenWrt SNAPSHOT r9945-bc85640cdc)
Thu May  9 11:02:08 2019 user.info banIP-[0.1.0]: start banIP processing (start)
Thu May  9 11:02:45 2019 user.info banIP-[0.1.0]: start banIP processing (reload)
Thu May  9 11:03:37 2019 user.info banIP-[0.1.0]: 21 IPSets with overall 172722 IPs/Prefixes loaded successfully (UBNT-ERX, OpenWrt SNAPSHOT r9945-bc85640cdc)
Thu May  9 11:04:10 2019 user.info banIP-[0.1.0]: start banIP processing (start)
Thu May  9 11:04:55 2019 user.info banIP-[0.1.0]: 21 IPSets with overall 172724 IPs/Prefixes loaded successfully (UBNT-ERX, OpenWrt SNAPSHOT r9945-bc85640cdc)
Thu May  9 11:06:28 2019 user.info banIP-[0.1.0]: start banIP processing (start)
Thu May  9 11:07:11 2019 user.info banIP-[0.1.0]: 21 IPSets with overall 172722 IPs/Prefixes loaded successfully (UBNT-ERX, OpenWrt SNAPSHOT r9945-bc85640cdc)
Thu May  9 11:08:45 2019 user.info banIP-[0.1.0]: start banIP processing (start)
Thu May  9 11:09:30 2019 user.info banIP-[0.1.0]: 21 IPSets with overall 172722 IPs/Prefixes loaded successfully (UBNT-ERX, OpenWrt SNAPSHOT r9945-bc85640cdc)
Thu May  9 11:11:02 2019 user.info banIP-[0.1.0]: start banIP processing (start)
Thu May  9 11:11:47 2019 user.info banIP-[0.1.0]: 21 IPSets with overall 172723 IPs/Prefixes loaded successfully (UBNT-ERX, OpenWrt SNAPSHOT r9945-bc85640cdc)
Thu May  9 11:12:49 2019 user.info banIP-[0.1.0]: start banIP processing (start)
Thu May  9 11:13:33 2019 user.info banIP-[0.1.0]: 21 IPSets with overall 172723 IPs/Prefixes loaded successfully (UBNT-ERX, OpenWrt SNAPSHOT r9945-bc85640cdc)```

That's the default - just add the missing file (see https://openwrt.org/docs/guide-user/base-system/cron for further information).

That's unexpected. banIP will be triggered by interface events (option 'ban_iface' in /etc/config/banip) and will be triggered by firewall reloads. Check the logs for such events.

BTW, the EdgerouterX is a really fine piece of hardware (IMHO)! :wink:

Thanks for the pointer regarding /etc/contabs/root. I am new to OpenWRT (though not new to Linux or networking), so still trying to learn the proper OpenWRT processes and best practices. :slight_smile:

So after your reply, I did some additional experimenting. If I only have wan selected for ban_iface (regardless of ban_automatic '0' or '1') I do not see the 'start banIP processing...' loop. However, if wan6 is enabled, with or without wan, the loop occurs.

Obviously this is related to something around wan6, but there are absolutely no other indications in the system log. Is it proper that I should have both wan and wan6 enabled, given that I operate in a dual-stack environment?

Any pointers as to what else I can could check?

P.S. - I agree wholeheartedly about the EdgerouterX--I am SO impressed by it! I thought maybe it would turn out to be under-powered, but it churns out some impressive capabilities. Even with SQM enabled (HW offloading disabled, obviously), it keeps up very nicely with my 480/24 mbps connection.

Yep, from my point of view it's a bug in OpenWrt/odhcpd - see unconfirmed ticket here: https://bugs.openwrt.org/index.php?do=details&task_id=1492

If trigger interface 'wan' works for you as well, than please use just that.

1 Like

Thanks for linking to that bug. I was not aware of that, but it could very well be what’s going on in my case.

So just confirming, the value for ban_iface is only defining interfaces that will trigger banIP reload, yeah? It does not affect banIP’s ability to actually block IPv4/IPv6 for any given Blocklist Source selected?

I really appreciate your time in responding. This has been a great learning experience for me!

yep, that's right - the trigger is used to bring up banIP on router boot and whenever the specified interface get's up (again).

1 Like

I use openwrt on a TpLink archer C6 v2 with Flash 8 MB and RAM 128 MB

I have two questions.

First I got an erros msg everytime I update the ipban lists:

*May 15 08:44:19 OpenWrt banIP-[0.1.0]: f_ipset ::: name: bogon, mode: create, settype: net, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 3044/0/3044, time(s): 10*
*May 15 08:44:20 OpenWrt banIP-[0.1.0]: f_ipset ::: name: myip_6, mode: create, settype: ip, setipv: inet6, ruletype: src+dst, count(sum/ip/cidr): 130/130/0, time(s): 10*
*May 15 08:44:22 OpenWrt banIP-[0.1.0]: f_ipset ::: name: debl_6, mode: create, settype: ip, setipv: inet6, ruletype: src+dst, count(sum/ip/cidr): 2/2/0, time(s): 12*
*May 15 08:44:28 OpenWrt banip.sh[19706]: sort: out of memory*
*May 15 08:44:30 OpenWrt banIP-[0.1.0]: f_ipset ::: name: myip, mode: create, settype: ip, setipv: inet, ruletype: src+dst, count(sum/ip/cidr): 6118/6118/0, time(s): 20*

Q1: How can i fix this ? The Flash is pretty full 89% but RAm space is only 20%

I use a central syslog server to capture all the logs on another machine. I Like to export the "IP-Set Lookup" logs as well.
Q2: how do i do that ?

kind regards

Maybe it's the best approach to remove the sort step in banIP.
For testing change line 681 in /usr/bin/banip.sh as follows:

old:
awk "${src_rset}" "${tmp_load}" 2>/dev/null | sort -u > "${tmp_file}"

new:
awk "${src_rset}" "${tmp_load}" 2>/dev/null > "${tmp_file}"

The sort step is pretty useless, cause ipset -! ignores duplicates anyway.
So give it a try please ...

1 Like

The IPSet lookup is currently only available in LuCI. The CLI part is on my todo list but currently I have no time for that ... for your own scripting, start with something like that:

ipset -L <IPSET> 2>/dev/null | egrep 'packets [1-9]'

Hope this helps.

1 Like

Dear Dibdot,

thx for the quick answer !
I Did change that particular line and the since then no more errors !

As my hardware is pretty new i do compile my own firmware quite often so the question I have is will you and when put this "fix" into the packages in the openwrt repro ?

kind regards

The IPSet lookup is currently only available in LuCI. The CLI part is on my todo list but
currently I have no time for that ... for your own scripting, start with something like that
ipset -L <IPSET> 2>/dev/null | egrep 'packets [1-9]'
Hope this helps.

mhh I hoped for something liked forward all ipset log to the remote syslog.

Or do i missed something here ?

When i run

set -L 2>/dev/null | egrep 'packets [1-9]'

I get the raw numbers but not the name of the list, to figure out with is the most effektive one.

regards