[Solved] 22.03.0: workaround for running CrowdSec bouncer

With installing OpenWrt 22.03.0 I realised that the crowdsec-firewall-bouncer package is not avaliable.

To fix this, I logged in into the router and downloaded the package form 21.02.3 branch here directly and installed manually.

wget https://downloads.openwrt.org/releases/21.02.3/packages/arm_cortex-a9_vfpv3-d16/packages/crowdsec-firewall-bouncer_0.0.21-3_arm_cortex-a9_vfpv3-d16.ipk
opkg install crowdsec-firewall-bouncer_0.0.21-3_arm_cortex-a9_vfpv3-d16.ipk

With this crowdsec showed up in UI in System -> Startup section.

Before starting I edited /etc/crowdsec/bouncer/crowdsec-firewall-bouncer.yaml. This is how mine is looking:

mode: nftables
pid_dir: /var/run/
update_frequency: 10s
daemonize: true
log_mode: file
log_dir: /var/log/
log_level: info
api_url: http://cs-lapi:8014/
api_key: ****************************
disable_ipv6: false
deny_action: DROP
deny_log: true
supported_decisions_types:
  - ban
#to change log prefix
#deny_log_prefix: "crowdsec: "
#to change the blacklists name
#blacklists_ipv4: crowdsec-blacklists
#blacklists_ipv6: crowdsec6-blacklists
#if present, insert rule in those chains
#iptables_chains:
#  - INPUT
#  - FORWARD
#  - DOCKER-USER
nftables:
  ipv4:
    enabled: true
    set-only: true
    table: crowdsec
    chain: crowdsec-chain
  ipv6:
    enabled: true
    set-only: true
    table: crowdsec6
    chain: crowdsec6-chain

You'll need to set your own values for api_url and api_key.

Now restart the bouncer in UI in System -> Startup.

The bouncer shall (or it shouldn't but it does) create tables for ip and ip6 with one chain each which are hooking into input path. I don't need this as this is not a server, but a router and the router wan is completely protected. So I modified the crowdsec tables with:

nft delete chain ip crowdsec crowdsec-chain
nft delete chain ip6 crowdsec6 crowdsec6-chain 

nft add chain ip crowdsec crowdsec-chain '{ type filter hook forward priority 4; policy accept; }'
nft add rule ip crowdsec crowdsec-chain iifname { wan, wg1 } ct state new ip saddr @crowdsec-blacklists log prefix \"crowdsec: \" counter drop

nft add chain ip6 crowdsec6 crowdsec6-chain '{ type filter hook forward priority 4; policy accept; }'
nft add rule ip6 crowdsec6 crowdsec6-chain iifname { wan, wg1 } ct state new ip6 saddr @crowdsec6-blacklists log prefix \"crowdsec: \" counter drop

I have the crowdsec filter working on wan and a wireguard interface. You may reduce the statements to wan only. Also, logging is done to syslog, you can check the bouncer working in UI.

This is how it looks in UI:

Hope it helps

3 Likes

Thanks for the instructions!

I created the api_key with the command: cscli bouncers add crowdsec-fire
wall-bouncer

and followed the rest of your instructions.

However, the nft rules do not get created for me. I get the following errors:

'nft delete chain ip crowdsec crowdsec-chain
Error: Could not process rule: No such file or directory
delete chain ip crowdsec crowdsec-chain
^^^^^^^^'

and:

'nft add chain ip crowdsec crowds
ec-chain '{ type filter hook forward priority 4; policy accept; }'
Error: No such file or directory
add chain ip crowdsec crowdsec-chain { type filter hook forward priority 4; policy accept; }
^^^^^^^^'

the ^^^^^^ are supposed to be below crowdsec in the code above.

Btw.: the config file you mentioned resides in: /etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml

Also: which ui is that at the end of your post? I cannot seem to find it or maybe I am missing something. Both crowdsec and bouncer services are up and running on my machine.

Was there not a name change on the bouncer?

For me this is still working. Updated to 22.03.02 today and reinstalled crowdsec bouncer.

The ui is the Firewall ui under Status (first menu entry).

Hello. I need some help:
With

nft delete chain ip crowdsec crowdsec-chain
nft delete chain ip6 crowdsec6 crowdsec6-chain 

nft add chain ip crowdsec crowdsec-chain '{ type filter hook forward priority 4; policy accept; }'
nft add rule ip crowdsec crowdsec-chain iifname { wan, wg1 } ct state new ip saddr @crowdsec-blacklists log prefix \"crowdsec: \" counter drop

nft add chain ip6 crowdsec6 crowdsec6-chain '{ type filter hook forward priority 4; policy accept; }'
nft add rule ip6 crowdsec6 crowdsec6-chain iifname { wan, wg1 } ct state new ip6 saddr @crowdsec6-blacklists log prefix \"crowdsec: \" counter drop

everything works great, but after reboot I loose the config.
How can I make it permanent?

Thanks :+1:

Geee, how'd you get so much identical rule entries?
The workaround gets lost on a reboot, yes. I'm currently trying to make a tweaked package for me that inserts rules on startup. I may share info when done.

1 Like

:smile: dont know... After I put your crowdsec tables comands, every thing shows and works good, wan and vpn...
Thanks for sharing your work :wink:

I'd just say, as the picture shows the crowdsec chain as a hook on the input. This is not what my rules create.

For me I'm trying to get a few things working and I will share my work when completed

  • I want to setup bridge filtering again as this ddin't work with 21.02
  • I want three filters (not interferring with fw4), those are
    • -bcp38- (done)
    • crowdsec bouncer in forward chain to my exposed systems
    • a filter not allowing traffic from internal networks to ips that has not been resolved by dnsmasq beforehand (plus managed exceptions)
1 Like

Hello :slightly_smiling_face:

That picture shown how it got after reboot.
Once I execute your rules, shows like this:

:wink: :+1:

Hello.
I edited /etc/firewall.cs with:

#!/bin/sh
/etc/init.d/crowdsec enabled && /etc/init.d/crowdsec restart
/etc/init.d/crowdsec-firewall-bouncer enabled && /etc/init.d/crowdsec-firewall-bouncer restart
nft delete chain ip crowdsec crowdsec-chain
nft add chain ip crowdsec crowdsec-chain '{ type filter hook forward priority 4; policy accept; }'
nft add rule ip crowdsec crowdsec-chain iifname { wan, AIRVPN2 } ct state new ip saddr @crowdsec-blacklists log prefix \"crowdsec: \" counter drop
service crowdsec reload
service crowdsec-firewall-bouncer reload
exit 0

Now I get this after reboot:


:+1:

1 Like

Hi
why'd you have

/etc/init.d/crowdsec enabled && /etc/init.d/crowdsec restart
service crowdsec reload

in there? Do you really have the crowdsec package installed? On my router it did not have enough space and the bouncer is quite enough if you have bouncer credentials already.

You have to use exroot:

I have crowdsec and AdguardHome installed :slightly_smiling_face:

Ok

But do you have a Crowdsec agent running on the OpenWrt device?

Yes.

And blocking IP´s from my "cscli decisions import -i /etc/crowdsec/ban.csv"

All working good.

Just waiting for v1.4.2 :slight_smile:

:+1:

This is a really useful thread, have installed Crowdsec + bouncer as per instructions (thank you!).
Particularly being able to persist the changes, thanks!

Just to add to anyone else following thread for first time, this location has examples of commands (such as viewing bans, listing installed / available plugins etc):

Seems a shame the way this other thread went, in terms of maintaining the new releases:

I've tried to build new packages for a wifi module using openWRT SDK and failed.
Perhaps will try and look into again in future.

1 Like

OpenWRT doesn't default to output syslog to file, so how does the crowdsec package scan for threats beyond importing IP bans from rest of world?

Default setup has this:

cscli metrics

INFO[15-12-2022 04:55:05 PM] Local Api Metrics:                           
+----------------------+--------+------+
|        ROUTE         | METHOD | HITS |
+----------------------+--------+------+
| /v1/alerts           | GET    |    7 |
| /v1/decisions/stream | GET    |  625 |
| /v1/watchers/login   | POST   |   10 |
+----------------------+--------+------+
INFO[15-12-2022 04:55:05 PM] Local Api Machines Metrics:                  
+--------------------------------------------------+------------+--------+------+
|                     MACHINE                      |   ROUTE    | METHOD | HITS |
+--------------------------------------------------+------------+--------+------+
| d26ffdf46b234e47a83324358a9aa308saAtnf4uLr27aJ3P | /v1/alerts | GET    |    7 |
+--------------------------------------------------+------------+--------+------+
INFO[15-12-2022 04:55:05 PM] Local Api Bouncers Metrics:                  
+------------------------------------+----------------------+--------+------+
|              BOUNCER               |        ROUTE         | METHOD | HITS |
+------------------------------------+----------------------+--------+------+
| crowdsec-firewall-bouncer-QUBnPT2q | /v1/decisions/stream | GET    |  625 |
+------------------------------------+----------------------+--------+------+

So I changed OpenWRT config to output syslog to a file. Now i get:

cscli metrics
INFO[15-12-2022 04:55:05 PM] Acquisition Metrics:                         
+----------------------+------------+--------------+----------------+------------------------+
|        SOURCE        | LINES READ | LINES PARSED | LINES UNPARSED | LINES POURED TO BUCKET |
+----------------------+------------+--------------+----------------+------------------------+
| file:/var/log/syslog |       1251 | -            |           1251 | -                      |
+----------------------+------------+--------------+----------------+------------------------+
INFO[15-12-2022 04:55:05 PM] Parser Metrics:                              
+---------------------------------+------+--------+----------+
|             PARSERS             | HITS | PARSED | UNPARSED |
+---------------------------------+------+--------+----------+
| child-crowdsecurity/syslog-logs | 2502 | -      |     2502 |
| crowdsecurity/syslog-logs       | 1251 | -      |     1251 |
+---------------------------------+------+--------+----------+
INFO[15-12-2022 04:55:05 PM] Local Api Metrics:                           
+----------------------+--------+------+
|        ROUTE         | METHOD | HITS |
+----------------------+--------+------+
| /v1/alerts           | GET    |    7 |
| /v1/decisions/stream | GET    |  625 |
| /v1/watchers/login   | POST   |   10 |
+----------------------+--------+------+
INFO[15-12-2022 04:55:05 PM] Local Api Machines Metrics:                  
+--------------------------------------------------+------------+--------+------+
|                     MACHINE                      |   ROUTE    | METHOD | HITS |
+--------------------------------------------------+------------+--------+------+
| d26ffdf46b234e47a83324358a9aa308saAtnf4uLr27aJ3P | /v1/alerts | GET    |    7 |
+--------------------------------------------------+------------+--------+------+
INFO[15-12-2022 04:55:05 PM] Local Api Bouncers Metrics:                  
+------------------------------------+----------------------+--------+------+
|              BOUNCER               |        ROUTE         | METHOD | HITS |
+------------------------------------+----------------------+--------+------+
| crowdsec-firewall-bouncer-QUBnPT2q | /v1/decisions/stream | GET    |  625 |
+------------------------------------+----------------------+--------+------+

Which makes "more sense" in that i can see syslog being read.
In terms of alerts, there are none (only the community blocklist)
Zero "Lines Parsed" from syslog:


root@OpenWrt:/etc/crowdsec# cscli alerts list
+----+-----------------------------------+------------------------+---------+----+-----------+-------------------------------+
| ID |               VALUE               |         REASON         | COUNTRY | AS | DECISIONS |          CREATED AT           |
+----+-----------------------------------+------------------------+---------+----+-----------+-------------------------------+
|  8 | crowdsecurity/community-blocklist | update : +32/-0 IPs    |         |    | ban:32    | 2022-12-15 15:27:20 +0000 UTC |
|  7 | crowdsecurity/community-blocklist | update : +15000/-0 IPs |         |    | ban:14981 | 2022-12-15 12:30:35 +0000 UTC |
|  6 | crowdsecurity/community-blocklist | update : +15000/-0 IPs |         |    | ban:53    | 2022-12-15 10:25:22 +0000 UTC |
|  5 | crowdsecurity/community-blocklist | update : +15000/-0 IPs |         |    | ban:348   | 2022-12-14 21:29:08 +0000 UTC |
|  4 | crowdsecurity/community-blocklist | update : +15000/-0 IPs |         |    | ban:61    | 2022-12-14 19:29:08 +0000 UTC |
|  3 | crowdsecurity/community-blocklist | update : +15000/-0 IPs |         |    | ban:58    | 2022-12-14 17:29:08 +0000 UTC |
|  2 | crowdsecurity/community-blocklist | update : +15000/-0 IPs |         |    | ban:6     | 2022-12-14 15:27:19 +0000 UTC |
|  1 | crowdsecurity/community-blocklist | update : +15000/-0 IPs |         |    | ban:12071 | 2022-12-14 13:24:21 +0000 UTC |
+----+-----------------------------------+------------------------+---------+----+-----------+-------------------------------+

As the title of this thread implies: this is a workaround to get the bouncer running on OpenWrt.

With my WRT3200ACM I don't have more space to install the crowdsec cscli package. It is also not needed for the bouncer. The bouncer just blocks the ips it retrieves from the local Crowdsec API.

As long as someone does not run some services that need protection on the OpenWrt device itself, the bouncer only is sufficiant. The usual setup is to have a local API running somewhere in the internal network plus a (few) agent(s) runnign on the services to protect. Those send decissions to the local API from where the bouncer retrieves this list.

Also: the router shall be protected by firewall rules to not accept connections from outside to the router itself. In this case, there is no need to protect the router with the Crowdsec bouncer. That's why the rules I posted are hooking into forward chain, protected the services inside the network by blocking the malicious connection attemps on the router already.

1 Like

I'm hoping to run Exim+Dovecot directly from the router, have setup the DDNS and within 24 hours can see attempts on port 22

What would I need to look at changing please? Change the hook to input?

Yes, if you run your services on the Openwrt device you need to protect the input hook.

I've set up my own mail server with postfix and dovecot on a device in dmz. Setting up and securing a mail server is some work to do.

1 Like

Interesting, was this using the openWRT packages? I ask because Postfix didn't look to be updated compared to Exim packages.

I will admit, had no idea setting up email server would be so tricky. So far I've got about 2 pages of A4 notes and when it started to dawn on me that mistakes will lead to big problems, thats when I started looking at trying to add Crowdsec.

Speaking of Crowdsec. I managed to use the 22.03.2 OpenWRT SDK and compile Crowdsec 1.4.3 (x86_64).
For what its worth, it does work with the cs-firewall-bouncer 0.0.21 that we've installed from openWRT 21.02.3 onto 22.03.2

But I could not repeat the same trick in terms of SDK compiling of cs-firewall-bouncer 0.0.24-rc1. It doesnt show up in make menuconfig and simply editing the .config and Makefiles doesnt get an ipk.
Going to park that one for a bit and come back to it given 1.4.3 is working with 0.0.21 anyway.

EDIT:

Just to confirm, I changed /etc/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml
from set-only true to false

nftables:
  ipv4:
    enabled: true
    set-only: false
    table: crowdsec
    chain: crowdsec-chain
  ipv6:
    enabled: true
    set-only: false
    table: crowdsec6
    chain: crowdsec6-chain

EDIT2:

On reboot, unfortunately the nftables are not populated with the IP bans. And cat /var/log/crowdsec-firewall-bouncer.log reports error "auth with api key failed" which apparently is due to bouncer starting before crowdsec (this happened on 1.3.0 too) . So I added "sleep 2" to /etc/firewall.cs (see below). A bit bodgy......but works :slight_smile:

Errors
time="15-12-2022 22:37:00" level=info msg="backend type : nftables"
time="15-12-2022 22:37:00" level=info msg="nftables initiated"
time="15-12-2022 22:37:00" level=info msg="Processing new and deleted decisions . . ."
time="15-12-2022 22:37:00" level=error msg="auth-api: auth with api key failed return nil response, error: dial tcp [::1]:8080: connect: connection refused"
time="15-12-2022 22:37:00" level=fatal msg="Get \"http://localhost:8080/v1/decisions/stream?startup=true\": dial tcp [::1]:8080: connect: connection refused"
time="15-12-2022 22:37:02" level=info msg="backend type : nftables"
time="15-12-2022 22:37:02" level=info msg="nftables initiated"
time="15-12-2022 22:37:02" level=info msg="Processing new and deleted decisions . . ."
time="15-12-2022 22:37:02" level=error msg="auth-api: auth with api key failed return nil response, error: dial tcp [::1]:8080: connect: connection refused"
time="15-12-2022 22:37:02" level=fatal msg="Get \"http://localhost:8080/v1/decisions/stream?startup=true\": dial tcp [::1]:8080: connect: connection refused"

Weirdly on re-boot, there are repeats of the same rules. I did read earlier in this thread someone else had same issue, solved by the modified /etc/firewall.cs with rule deletions; more bodges :slight_smile:

Duplicate rules showing:

nft list table crowdsec
table ip crowdsec {
	set crowdsec-blacklists {
		type ipv4_addr
		flags timeout
	}

	chain crowdsec-chain {
		type filter hook input priority filter; policy accept;
		ip saddr @crowdsec-blacklists drop
		ip saddr @crowdsec-blacklists drop
		ip saddr @crowdsec-blacklists drop
		ip saddr @crowdsec-blacklists drop
	}
}

My /etc/firewall.cs now looks like this

#!/bin/sh
#/etc/init.d/crowdsec enabled && /etc/init.d/crowdsec restart
#/etc/init.d/crowdsec-firewall-bouncer enabled && /etc/init.d/crowdsec-firewall-bouncer restart
#nft delete chain ip crowdsec crowdsec-chain
#nft delete chain ip6 crowdsec6 crowdsec6-chain
#nft add chain ip crowdsec crowdsec-chain '{ type filter hook forward priority 4; policy accept; }'
#nft add rule ip crowdsec crowdsec-chain iifname { usb0, eth4 } ct state new ip saddr @crowdsec-blacklists log prefix \"crowdsec: \" counter drop
#nft add chain ip6 crowdsec6 crowdsec6-chain '{ type filter hook forward priority 4; policy accept; }'
#nft add rule ip6 crowdsec6 crowdsec6-chain iifname { usb0, eth4 } ct state new ip6 saddr @crowdsec6-blacklists log prefix \"crowdsec: \" counter drop
#service crowdsec reload
sleep 2
service crowdsec-firewall-bouncer reload
sleep 2
nft delete rule crowdsec crowdsec-chain handle 6
nft delete rule crowdsec crowdsec-chain handle 5
nft delete rule crowdsec crowdsec-chain handle 4
nft delete rule ip6 crowdsec6 crowdsec6-chain handle 6
nft delete rule ip6 crowdsec6 crowdsec6-chain handle 5
nft delete rule ip6 crowdsec6 crowdsec6-chain handle 4
exit 0

So now on bootup

cat /var/log/crowdsec-firewall-bouncer.log shows

time="15-12-2022 23:08:25" level=info msg="backend type : nftables"
time="15-12-2022 23:08:25" level=info msg="nftables initiated"
time="15-12-2022 23:08:25" level=info msg="Processing new and deleted decisions . . ."
time="15-12-2022 23:09:38" level=info msg="'1352' decisions deleted"
time="15-12-2022 23:09:39" level=info msg="'26261' decisions added"

And nft list table crowdsec has data + no duplicate rule.

			     223.197.145.33 timeout 2d17h15m55s140ms expires 2d17h1s10ms, 223.197.151.55 timeout 4d18h15m55s130ms expires 4d18h1s200ms,
			     223.197.153.135 timeout 4d10h22m9s130ms expires 4d10h6m15s140ms, 223.197.175.91 timeout 5d12h15m55s120ms expires 5d12h1s490ms,
			     223.197.186.7 timeout 5d3h15m55s130ms expires 5d3h1s280ms, 223.197.188.206 timeout 5d12h15m55s130ms expires 5d12h1s410ms,
			     223.197.208.186 timeout 4d19h22m9s130ms expires 4d19h6m15s230ms, 223.197.220.67 timeout 6d10h22m9s120ms expires 6d10h6m15s710ms,
			     223.240.83.206 timeout 5d12h15m55s130ms expires 5d12h1s460ms, 223.240.96.1 timeout 5d12h15m55s130ms expires 5d12h1s470ms,
			     223.241.222.151 timeout 13h22m9s140ms expires 13h6m14s870ms, 223.245.0.5 timeout 5d12h15m55s120ms expires 5d12h1s460ms }
	}

	chain crowdsec-chain {
		type filter hook input priority filter; policy accept;
		ip saddr @crowdsec-blacklists drop
	}
}
root@OpenWrt:~# 

1 Like