OpenWRT 25.12 - RSyslog - Segfault

I just upgraded to 25.12 with a working RSyslog configurationin 24.10, which is now segfaulting.

Note: I use rsyslog as a replacement for the log facility in OpenWRT. To be honest, I dont remember the reason why I did this (this is a centralized log server for various devices)

root@OpenWrt-filogic:~# service | grep log
/etc/init.d/log                   disabled         stopped
/etc/init.d/rsyslog                enabled         stopped
root@OpenWrt-filogic:~# grep -v ^# /etc/rsyslog.conf 
global(net.ipprotocol="ipv4-only")

template(
  name="UTCTraditionalForwardFormat"
  type="string"
  string="<%PRI%>%TIMESTAMP:::date-utc% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%"
)
root@OpenWrt-filogic:~# cat /etc/config/rsyslog 

config syslog 'syslog'
        option tcp_input_port '514'
        option udp_input '1'
        option tcp_input '1'
        option udp_input_port '514'
        option default_template 'RSYSLOG_TraditionalFileFormat'
        list modules 'imuxsock'
        list modules 'imklog'

config selector
        option source '*.info;mail.none;authpriv.none;cron.none'
        option destination '/mnt/sda2/var/log/messages'

config selector
        option source 'authpriv.*'
        option destination '/mnt/sda2/var/log/secure'

config selector
        option source 'mail.*'
        option destination '/mnt/sda2/var/log/maillog'

config selector
        option source 'cron.*'
        option destination '/mnt/sda2/var/log/cron'

config selector
        option source 'local7.*'
        option destination '/mnt/sda2/var/log/boot.log'
root@OpenWrt-filogic:~# strace /usr/sbin/rsyslogd -f /var/etc/rsyslog.conf -n
execve("/usr/sbin/rsyslogd", ["/usr/sbin/rsyslogd", "-f", "/var/etc/rsyslog.conf", "-n"], 0x7fe0b47668 /* 14 vars */) = 0
<snip>
--- SIGTTOU {si_signo=SIGTTOU, si_code=SI_USER, si_pid=14032, si_uid=0} ---
rt_sigreturn({mask=[]})                 = 0
openat(AT_FDCWD, "/var/run/rsyslogd.pid.tmp", O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 10
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f86051000
ioctl(10, TIOCGWINSZ, 0x7fc46179c8)     = -1 ENOTTY (Not a tty)
writev(10, [{iov_base="14032", iov_len=5}, {iov_base=NULL, iov_len=0}], 2) = 5
close(10)                               = 0
munmap(0x7f86051000, 4096)              = 0
renameat(AT_FDCWD, "/var/run/rsyslogd.pid.tmp", AT_FDCWD, "/var/run/rsyslogd.pid") = 0
kill(14032, SIGTTOU)                    = 0
--- SIGTTOU {si_signo=SIGTTOU, si_code=SI_USER, si_pid=14032, si_uid=0} ---
rt_sigreturn({mask=[]})                 = 0
rt_sigprocmask(SIG_BLOCK, [HUP TERM CHLD], [], 8) = 0
socket(AF_UNIX, SOCK_DGRAM|SOCK_CLOEXEC, 0) = 10
connect(10, {sa_family=AF_UNIX, sun_path="/dev/log"}, 12) = 0
sendto(10, "<46>Dec 26 15:58:06 : [origin so"..., 125, 0, NULL, 0) = 125
futex(0x7f8743c338, FUTEX_WAIT_PRIVATE, 2147483650, NULL) = -1 EAGAIN (Resource temporarily unavailable)
pselect6(0, NULL, NULL, NULL, {tv_sec=600, tv_nsec=0}, {sigmask=[], sigsetsize=8} <unfinished ...>) = ?
+++ killed by SIGSEGV +++
Segmentation fault

Perhaps you have to do some magic to create /dev/log on each boot? It looks like that is missing from the strace.

I see /dev/log does exists, below is the device and ownership

root@OpenWrt-filogic:~# ls -ltra /dev/log
srw-rw-rw-    1 root     root             0 Dec 26 09:58 /dev/log

Strace in debug mode. Removed IP address on last line.

select6(0, NULL, NULL, NULL, {tv_sec=597, tv_nsec=416826914}, {sigmask=[], sigsetsize=8}) = ? ERESTARTNOHAND (To be restarted if no handler)
--- SIGWINCH {si_signo=SIGWINCH, si_code=SI_KERNEL} ---
pselect6(0, NULL, NULL, NULL, {tv_sec=597, tv_nsec=373719357}, {sigmask=[], sigsetsize=8}) = ? ERESTARTNOHAND (To be restarted if no handler)
--- SIGWINCH {si_signo=SIGWINCH, si_code=SI_KERNEL} ---
pselect6(0, NULL, NULL, NULL, {tv_sec=597, tv_nsec=288120113}, {sigmask=[], sigsetsize=8}) = ? ERESTARTNOHAND (To be restarted if no handler)
--- SIGWINCH {si_signo=SIGWINCH, si_code=SI_KERNEL} ---
pselect6(0, NULL, NULL, NULL, {tv_sec=597, tv_nsec=267845013}, {sigmask=[], sigsetsize=8}) = ? ERESTARTNOHAND (To be restarted if no handler)
--- SIGWINCH {si_signo=SIGWINCH, si_code=SI_KERNEL} ---
pselect6(0, NULL, NULL, NULL, {tv_sec=597, tv_nsec=247463944}, {sigmask=[], sigsetsize=8}) = ? ERESTARTNOHAND (To be restarted if no handler)
--- SIGWINCH {si_signo=SIGWINCH, si_code=SI_KERNEL} ---
pselect6(0, NULL, NULL, NULL, {tv_sec=597, tv_nsec=226055527}, {sigmask=[], sigsetsize=8}) = ? ERESTARTNOHAND (To be restarted if no handler)
--- SIGWINCH {si_signo=SIGWINCH, si_code=SI_KERNEL} ---
pselect6(0, NULL, NULL, NULL, {tv_sec=597, tv_nsec=204946226}, {sigmask=[], sigsetsize=8}) = ? ERESTARTNOHAND (To be restarted if no handler)
--- SIGWINCH {si_signo=SIGWINCH, si_code=SI_KERNEL} ---
pselect6(0, NULL, NULL, NULL, {tv_sec=597, tv_nsec=183996187}, {sigmask=[], sigsetsize=8}0814.494485429:imtcp.c        : tcpsrv.c: epoll_wait returned 1 entries
0814.494518004:imtcp.c        : tcpsrv.c: tcpsrv: ready to process 1 event entries
0814.494541414:imtcp.c        : tcpsrv.c: doing epoll_wait for max 128 events
0814.494584077:w0/imtcp       : tcpsrv.c: tcpsrv: processing item 0, socket 6
0814.494598016:w0/imtcp       : tcpsrv.c: New connect on NSD 0x7fab9f2800.
0814.494676873:w0/imtcp       : dnscache.c: findEntry: 1st lookup found 0
0814.494687423:w0/imtcp       : dnscache.c: findEntry: 2nd lookup found 0
0814.495265142:w0/imtcp       : tcpsrv.c: adding epoll entry 0, socket 12
0814.495298872:w0/imtcp       : tcpsrv.c: Done processing epoll entry 0, iRet 0
0814.495308729:w0/imtcp       : tcpsrv.c: New session created with NSD 0x7fabac50c0.
0814.495310654:imtcp.c        : tcpsrv.c: epoll_wait returned 1 entries
0814.495322437:w0/imtcp       : tcpsrv.c: New connect on NSD 0x7fab9f2800.
0814.495328751:imtcp.c        : tcpsrv.c: tcpsrv: ready to process 1 event entries
0814.495348389:imtcp.c        : tcpsrv.c: doing epoll_wait for max 128 events
0814.495354703:w0/imtcp       : tcpsrv.c: tcpsrv: processing item 0, socket 12
0814.495376035:w0/imtcp       : tcpsrv.c: netstream 0x7fab9f2940 with new data from remote peer 192.168.X.YYY
 <unfinished ...>) = ?
+++ killed by SIGSEGV +++
Segmentation fault
root@OpenWrt-filogic:~# 

Sorry, I missed the last line of the previous strace. The segfault appears to be occuring when receiving data from another host. I don't suppose you can enable core dumping and get a stack backtrace?

Either way, do you use plain (no SSL) TCP transport, or UDP, or RELP, or which to get the data to the system?

@cshoredaniel , I was able to switch over everything to use UDP, and it working fine. Thank you for giving me a kick on the right direction.

1 Like

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.