Syslog-ng: How to use logs normally and just remove banip logs and keep last * lines instead of log rotate

RSHARM · March 15, 2025, 8:20am

1. How to configure syslog-ng only remove banip line using filter banip and show other logs normally ? {

filter banip {
	not match("banIP")
};

2. Instead of log rotate how keep last 2000 lines like normal ?

3.

banIP support thread

log {
	source(src);
	source(net);
	source(kernel);
	source(s_network);
	filter(banip);
	filter(collectd);
	filter(firewall_lan_fwd);
	filter(firewall_lan2_fwd);
	filter(firewall_wan_drp);
	filter(firewall_wan_in);
	filter(firewall_wan_fwd);
	filter(openvpn_fail);
	filter(openvpn_info);
	destination(messages);
};

How does this part work ?

lleachii · March 15, 2025, 10:49am

Syslog-ng isn't the default OpenWrt logging software, so here is the link to the general documentation:

Note the configuration file and filter sections
Tagging @acid8000 to chime in

acid8000 · March 15, 2025, 1:11pm

Here's my full /etc/syslog-ng.conf for context:

# Collect all local logs into a single file /var/log/messages.
# See https://www.syslog-ng.com/technical-documents/list/syslog-ng-open-source-edition

@version: 4.7
@include "scl.conf"

options {
        chain_hostnames(no); # Enable or disable the chained hostname format.
        create_dirs(yes);
        keep_hostname(yes); # Enable or disable hostname rewriting.
        log_fifo_size(256); # The number of messages that the output queue can store.
        log_msg_size(1024); # Maximum length of a message in bytes.
        stats(freq(0)); # The period between two STATS messages (sent by syslog-ng, containing statistics about dropped logs) in seconds.
        flush_lines(0); # How many lines are flushed to a destination at a time.
        use_fqdn(no); # Add Fully Qualified Domain Name instead of short hostname.
};

# syslog-ng gets messages from syslog-ng (internal) and from /dev/log

source src {
        internal();
        unix-dgram("/dev/log");
};

source net {
        network_localhost();
};

source s_network {
        udp(ip(0.0.0.0) port(514));
};

source kernel {
        file("/proc/kmsg" program_override("kernel"));
};

filter banip {
        not match("banIP")
};

filter collectd {
        not match("collectd") and
        not match("MyCloudPR4100")
};

filter firewall_lan_fwd {
        not match("reject lan forward")
};

filter firewall_lan2_fwd {
        not match("reject lan2 forward")
};

filter firewall_wan_drp {
        not match("drop wan invalid")
};

filter firewall_wan_in {
        not match("reject wan in")
};

filter firewall_wan_fwd {
        not match("reject wan forward")
};

filter openvpn_fail {
        not match("read UDPv6") and
        not match("NO-INFO") and
        not match("Connection refused")
};

filter openvpn_info {
        not match("peer info")
};

destination messages {
        file("/var/log/messages");
};

log {
        source(src);
        source(net);
        source(kernel);
        source(s_network);
        filter(banip);
        filter(collectd);
        filter(firewall_lan_fwd);
        filter(firewall_lan2_fwd);
        filter(firewall_wan_drp);
        filter(firewall_wan_in);
        filter(firewall_wan_fwd);
        filter(openvpn_fail);
        filter(openvpn_info);
        destination(messages);
};

# Include any user settings last so that s/he can override or
# supplement all "canned" settings inherited from the distribution.
@include "/etc/syslog-ng.d/" # Put any customization files in this directory

I just modified the default config file for the above. The filters are for banIP and firewall so that they only appear in the kernel log, and to eliminate useless (for me) OpenVPN and collectd entries entirely. I'm unaware whether syslog-ng supports limiting the syslog to x number of entries like the stock OpenWRT logging tool, but saw recommendations to install and use logrotate to prevent it from causing an out of memory problem. My router is doing logging for my entire network for all client devices that support logging to a remote device.

That last snippet of the config at the top of this post combines the different sources, passes them through the filters, then sends what's left to the syslog.

RSHARM · March 16, 2025, 6:09am

I used your config and it says Error: logfile not found!

acid8000 · March 16, 2025, 8:32am

I don't recall how I fixed that problem nor if I did experience it at all, but try one of two things:

Create the file manually:

nano /var/log/messages

Force syslog-ng to restart:

killall syslog-ng
/etc/init.d/syslog-ng start

RSHARM · March 16, 2025, 6:41pm

https://192.168.1.1/cgi-bin/luci/admin/status/logs/syslog is empty and also https://192.168.1.1/cgi-bin/luci/admin/services/banip/firewall_log

lleachii · March 16, 2025, 7:38pm

I think you forgot:

pavelgl · March 16, 2025, 7:55pm

This means that there is some kind of error in the configuration file and the service doesn't start correctly. Post the full contents of /etc/syslog-ng.conf.

RSHARM · March 17, 2025, 9:45am

# Collect all local logs into a single file /var/log/messages.
# See https://www.syslog-ng.com/technical-documents/list/syslog-ng-open-source-edition

@version: 4.7
@include "scl.conf"

options {
        chain_hostnames(no); # Enable or disable the chained hostname format.
        create_dirs(yes);
        keep_hostname(yes); # Enable or disable hostname rewriting.
        log_fifo_size(256); # The number of messages that the output queue can store.
        log_msg_size(1024); # Maximum length of a message in bytes.
        stats(freq(0)); # The period between two STATS messages (sent by syslog-ng, containing statistics about dropped logs) in seconds.
        flush_lines(0); # How many lines are flushed to a destination at a time.
        use_fqdn(no); # Add Fully Qualified Domain Name instead of short hostname.
};

# syslog-ng gets messages from syslog-ng (internal) and from /dev/log

source src {
        internal();
        unix-dgram("/dev/log");
};

source net {
        network_localhost();
};

source s_network {
        udp(ip(0.0.0.0) port(514));
};

source s_network {
	default-network-drivers(
		# NOTE: TLS support
		#
		# the default-network-drivers() source driver opens the TLS
		# enabled ports as well, however without an actual key/cert
		# pair they will not operate and syslog-ng would display a
		# warning at startup.
		#
		#tls(key-file("/path/to/ssl-private-key") cert-file("/path/to/ssl-cert"))
	);
};


source kernel {
        file("/proc/kmsg" program_override("kernel"));
};

filter banip {
        not match("banIP")
};

filter collectd {
        not match("collectd") and
        not match("MyCloudPR4100")
};

filter firewall_lan_fwd {
        not match("reject lan forward")
};

filter firewall_lan2_fwd {
        not match("reject lan2 forward")
};

filter firewall_wan_drp {
        not match("drop wan invalid")
};

filter firewall_wan_in {
        not match("reject wan in")
};

filter firewall_wan_fwd {
        not match("reject wan forward")
};

filter openvpn_fail {
        not match("read UDPv6") and
        not match("NO-INFO") and
        not match("Connection refused")
};

filter openvpn_info {
        not match("peer info")
};

destination messages {
        file("/var/log/messages");
};

log {
	source(src);			#default
	source(net);			#default
	source(kernel);			#default
	destination(messages);		#default
	source(s_network);		#default
	filter(banip);
	filter(collectd);
	filter(firewall_lan_fwd);
	filter(firewall_lan2_fwd);
	filter(firewall_wan_drp);
	filter(firewall_wan_in);
	filter(firewall_wan_fwd);
	filter(openvpn_fail);
	filter(openvpn_info);

};

# Include any user settings last so that s/he can override or
# supplement all "canned" settings inherited from the distribution.
@include "/etc/syslog-ng.d/" # Put any customization files in this directory

Also I created /var/log/messages using Win-SCP and it automatically got deleted

pavelgl · March 17, 2025, 10:08am

The s_network source is declared twice.

Comment out or delete this part:

By the way, you only need the banip filter.

filter banip {
        not match("banIP")
};

log {
        source(src);
        source(net);
        source(kernel);
        filter(banip);
        destination(messages);
};

RSHARM · March 17, 2025, 2:46pm

Also I created /var/log/messages using Win-SCP and it automatically got deleted

pavelgl · March 17, 2025, 4:25pm

Make the suggested changes and restart the service or reboot the router.

You do not need to create the log file manually. If everything is OK, it will be created automatically.

RSHARM · March 20, 2025, 2:15pm

Error: logfile not found!

pavelgl · March 20, 2025, 4:34pm

mv /etc/syslog-ng.conf /etc/syslog-ng.conf.bak

cat << "EOF" > /etc/syslog-ng.conf
@version: 4.7
@include "scl.conf"

options {
        chain_hostnames(no);
        create_dirs(yes);
        keep_hostname(yes); 
        log_fifo_size(256); 
        log_msg_size(1024);
        stats(freq(0));
        flush_lines(0);
        use_fqdn(no);
        keep_timestamp(no);
};

source src {
        internal();
        unix-dgram("/dev/log");
};

source net {
        network_localhost();
};

source s_network {
        default-network-drivers();  
        };

source kernel {
        file("/proc/kmsg" program_override("kernel"));
};

destination messages {
        file("/var/log/messages");
};

filter banip {
        not match("banIP")
};

log {
        source(src);
        source(net);
        source(kernel);
        filter(banip);
        destination(messages);
};


@include "/etc/syslog-ng.d/"
EOF

rm /var/log/messages*
/etc/init.d/syslog-ng restart
logread

RSHARM · March 21, 2025, 6:49am

Error: logfile not found!

RSHARM · March 22, 2025, 12:10pm

I uninstalled syslog-ng anfd https://192.168.1.1/cgi-bin/luci/admin/status/logs/syslog shows Unable to load log data: Exec permission denied

spence · March 22, 2025, 4:29pm

I have not used syslog-ng on OpenWrt but it is a different logging system than what is provided in the normal firmware.

syslog-ng likely is not a drop-in replacement for the built-in logging system. Do not expect interacting with syslog-ng logs to be the same as the built-in logging software unless explicitly stated in the documentation for the OpenWrt version of syslog-ng.
logread not working makes sense due to this.
The Luci page for logging not working makes sense due to this.

I think syslog-ng is more of an enterprise level logging system and will have corresponding use and management skills required.

The syslog-ng installer may or may not have disabled the built-in logging system and/or changed the system config.

Uninstalling syslog-ng may not have restored any pre-existing logging config and re-enabled the built-in logging system. You may need to review/restore the previous logging config from a backup and re-enable re-start service log.

pavelgl · March 22, 2025, 7:13pm

No one knows what you're doing except posting the same error message over and over again. I personally tested the suggested syslog-ng configuration on 24.10 and I am sure it works.

Did you remove logd before installing syslog-ng? Did you reinstall logd after removing syslog-ng?

RSHARM · March 22, 2025, 7:16pm

Did you remove logd before installing syslog-ng ?

No

Did you reinstall logd after removing sys

No but it's there

Also

Ramon · March 23, 2025, 11:45am

as pointed out already, it is either syslog-ng or logd. So to fix it, remove both. Then install the one you want