Log filtering (using rsyslog for local logging)

My system log, as displayed by LuCI, is often filled with hostapd STA-OPMODE-SMPS-MODE-CHANGED entries. I wanted to filter these messages. The below method forwards the (default) logd output to rsyslogd, configured to filter the log messages and write it to a local file. logrotate is used to limit the size of the generated log file, and LuCI is modified to display the system log from rsyslogd.

SSH into the router and install the necessary packages:

opkg update
opkg install rsyslog logrotate zoneinfo-all

Note: zoneinfo-all is there to get local time in the log. Instead of zoneinfo-all choose a zoneinfo-* package specific to the time zone your router is using.

Append the default rsyslog.conf to use a template similar to the logd format and to filter the log entries:

cat >>/etc/rsyslog.conf <<EOF
\$template myTemplate,"%timereported:::date-wdayname% %timereported% %timereported:::date-year% %syslogtag%%msg%\n"
:msg, contains, "STA-OPMODE-SMPS-MODE-CHANGED" stop
:msg, contains, "logrotate-rsyslog.conf" stop
:msg, contains, "rsyslogd was HUPed" stop
EOF

The first filter removes the STA-OPMODE-SMPS-MODE-CHANGED log entries, the second suppresses the crontab log entries related to running logrotate, the third one suppresses the log entries created when logrotate restarts rsyslogd.

Modify the rsyslog configuration so that rsyslog will use the template (myTemplate) added to rsyslog.conf and listens to log messages on the localhost port 515:

cat >/etc/config/rsyslog <<EOF
config syslog 'syslog'
        option tcp_input_port '515'
        option udp_input '1'
        option tcp_input '0'
        option udp_input_port '515'
        option default_template 'myTemplate'
        list modules 'imuxsock'
        list modules 'imklog'

config selector
        option source '*.*'
        option destination '/var/log/messages'
EOF

Enable and start rsyslog:

service rsyslog enable
service rsyslog start

Create a command for reading the log created by rsyslog:

cat >/sbin/logreadrsyslog <<EOF
#!/bin/sh
[ -f /var/log/messages.1 ] && cat /var/log/messages.1
[ -f /var/log/messages ] && cat /var/log/messages
EOF
chmod 755 /sbin/logreadrsyslog

Replace the logread command call with logreadrsyslog for LuCI in /www/luci-static/resources/view/status/syslog.js and enable the use of logreadrsyslog instead of logread in /usr/share/rpcd/acl.d/luci-mod-status.json (this is working on 23.05):

sed -i 's/logread/logreadrsyslog/g' /www/luci-static/resources/view/status/syslog.js
sed -i 's/logread/logreadrsyslog/g' /usr/share/rpcd/acl.d/luci-mod-status.json

Restart rcpd and uhttpd:

/etc/init.d/rpcd restart
/etc/init.d/uhttpd stop
rm -rf /tmp/luci*
/etc/init.d/uhttpd start

Configure the system logging so the log entries are sent to the localhost port 515 (where rsyslog is listening):

uci set system.@system[0].log_ip="127.0.0.1"
uci set system.@system[0].log_port="515"
uci commit system
/etc/init.d/log restart

Create a logrotate configuration to limit the size of the system log. This one limits the log size to a maximum of 2 files, 64kB each:

cat >/etc/logrotate.d/logrotate-rsyslog.conf <<EOF
/var/log/messages {
  rotate 1
  size 64k
  postrotate
    /usr/bin/killall -HUP rsyslogd
  endscript
  missingok
  nocompress
}
EOF

Add the following entry to your crontab (LuCI System / Scheduled Tasks) to run logrotate hourly:

0 * * * * /usr/sbin/logrotate /etc/logrotate.d/logrotate-rsyslog.conf

Note: on my system after I apply these changes the log entries start to use UTC time, but after a router reboot or after changing the timezone (using LuCI System / General Settings) the log starts using the local time.

Comments/suggestions are welcome!

1 Like

Looks very useful. Thanks for sharing all the details.

Using this template in rsyslog.conf also includes the syslog facility and severity:

$template myTemplate,"%timereported:::date-wdayname% %timereported% %timereported:::date-year% %syslogfacility-text%.%syslogseverity-text% %syslogtag%%msg%\n"

i tried your tuts, but ended up with "Unable to load log data: Access to command denied by ACL"

I suspect a browser caching issue. Have you tried opening the luci page in incognito mode?

If the same ACL issue occurs in incognito mode as well, then double check if logreadrsyslog is listed in /usr/share/rpcd/acl.d/luci-mod-status.json in the "luci-mod-status-logs" section. Also try to restart rcpd and uhttpd.

btw do we need to uninstall logd ?

There is no need to uninstall logd, I would rather keep it.