Syslog-ng log grows until forever -> device crash

[Catfriend1]

Hi,

I'm on Archer C7v2|5. But my problem is not device-specific. I've opkg removed "logd" and installed "syslog-ng". So far so good. This is my config - except the "0.0.0.0" part it's default config shipped via opkg.

One of my devices collects its own syslog and the syslog sent via other OpenWrt devices. Running for x days, the log grows without a limit.

ll /var/log/messages
-rw------- 1 root root 4231864 Mar 18 11:46 /var/log/messages

I've found via htop the syslog-ng is the culprit. The device has ~ 10 MByte RAM allocation, half for syslog-ng and the other half for the memory-backed "/var/log/messages".

How can I limit the log size in a way that syslog-ng will start dropping the oldest log entries when the limit is reached? Anyone out there who also uses syslog-ng this way who can give me advice?

Thanks. Kind regards,
Catfriend1

/etc/syslog-ng.conf

#############################################################################
# OpenWrt syslog-ng.conf specific file
# which collects all local logs into a single file called /var/log/messages.
# More details about these settings can be found here:
# https://www.syslog-ng.com/technical-documents/list/syslog-ng-open-source-edition

@version: 3.29
@include "scl.conf"

options {
	chain_hostnames(no); # Enable or disable the chained hostname format.
	create_dirs(yes);
	keep_hostname(yes); # Enable or disable hostname rewriting.
	log_fifo_size(256); # The number of messages that the output queue can store.
	log_msg_size(1024); # Maximum length of a message in bytes.
	stats_freq(0); # The period between two STATS messages (sent by syslog-ng, containing statistics about dropped logs) in seconds.
	flush_lines(0); # How many lines are flushed to a destination at a time.
	use_fqdn(no); # Add Fully Qualified Domain Name instead of short hostname.
	keep_timestamp(no);
};

# syslog-ng gets messages from syslog-ng (internal) and from /dev/log

source src {
	internal();
	unix-dgram("/dev/log");
};

source net {
	network(ip("0.0.0.0") port(514) transport(udp) ip-protocol(6));
};

source s_network {
	default-network-drivers(
		# NOTE: TLS support
		#
		# the default-network-drivers() source driver opens the TLS
		# enabled ports as well, however without an actual key/cert
		# pair they will not operate and syslog-ng would display a
		# warning at startup.
		#
		#tls(key-file("/path/to/ssl-private-key") cert-file("/path/to/ssl-cert"))
	);
};

source kernel {
        file("/proc/kmsg" program_override("kernel"));
};

destination messages {
	file("/var/log/messages");
};

log {
	source(src);
	source(net);
        source(kernel);
	destination(messages);

	# uncomment this line to open port 514 to receive messages
	#source(s_network);
};

#
# Finally, include any user settings last so that s/he can override or
# supplement all "canned" settings inherited from the distribution.
#
@include "/etc/syslog-ng.d/" # Put any customization files in this directory

[dl12345]

Use the logrotate package

[Catfriend1]

Ok, I thought this patch would be possible but it's not included yet.

https://lists.balabit.hu/pipermail/syslog-ng/2011-July/017011.html

[dl12345]

Install logrotate, add the file /etc/logrotate.d/messages

/var/log/messages {
   missingok
   rotate 7
   daily
   postrotate
	 /usr/sbin/syslog-ng-ctl reopen
   endscript
}

This rotates it once a day and keeps 7 days of logs. Tune as needed.

[Catfriend1]

Will do this, thank you @dl12345

[Catfriend1]

Follow up to: https://forum.openwrt.org/t/syslog-ng-log-grows-until-forever-device-crash

@dl12345

I've now set it up as recommended. I've scripts that rely on "logread -f" outputting new log lines permanently. To get this go on correctly after log rotation, I've added the option "copytruncate".

I did my tests with:

logrotate -f /etc/logrotate.d/messages

How does the OS know when/how to execute "logrotate" for the configured "messages" file once a day? Do I need a cron job for this? Could you please post me how you did this as an example? Thank you very much for your help.

My current settings file for logrotate:

/var/log/messages {
   missingok
   rotate 7
   daily
   copytruncate
   compress
   postrotate
	 /usr/sbin/syslog-ng-ctl reopen
   endscript
}

[frollic]

Yes,

logrotate is usually triggered from cron.

Scheduling tasks

[Catfriend1]

Thanks, I've added the following line to my "/etc/crontab/root" file:

0 0 * * * /usr/sbin/logrotate /etc/logrotate.conf >/dev/null 2>&1

Source of information: https://openwrt-users.openwrt.narkive.com/qFGKZeLp/crontab-in-openwrt