How to determine the reason for a sudden reboot of the router

Installing and Using OpenWrt
1

Hi everyone!

I have a fairly heavily loaded router, and with an additional increase in the load on it, it begins to reboot spontaneously approximately once every two hours.

What is the way to figure out why it happens? Maybe some way of logging kernel logs to the attached HDD or some way over the network?

My setup: Linksys EA8500 + 6 TB SATA HDD + 1GB swap partition based on USB-SATA SSD.

Services:

AdBlock with a 500k list of blocked domains
stubby (encrypting of DNS traffic)
ksmbd
rtorret with 200 torrents + rutorrent
minidlna (approximately 100k songs, 80k photos, 6k videos)
wireguard server
ddns client
nlbw
luci-app-statistics + collectd

You can see spontaneous reboots in the system load graph provided:

image
image966×281 9.06 KB

2

OOM
over heating
power issues

pick one

and yes, you can log to local storage, or over the network.

1 Like
3

I don't think reboots are caused by overheating, the router is in the garage and has an additional 30mm FAN:

image
image966×281 6.02 KB

OOM..... Why does it happen? I have plenty of free swap space.

Power issues - hm... Okay, I'll try another more power power supply, thanks for the suggestion.

root@OpenWrt:~# free -h
              total        used        free      shared  buff/cache   available
Mem:         476464      229544       70980       18612      175940      175532
Swap:       1048572           0     1048572

Could you please point me to how to configure the storage of kernel logs to the file?

4

The graph actually suggests otherwise...
Note that after most of the reboots there is a decrease from a really high temp. From a much higher level than your earlier "normal" running temp.

image
image814×253 84 KB

I suspect that some service process gets stuck into 100% CPU consumption and causes overheating.

2 Likes
5

what @hnyman said (nice spotting!), plus, a 30mm fan won't move a lot of air.

fan set up and location might be significant too, push-pull, pull-pull, push-push ?

if you're running some kind of soft RAID across those drives, it'll consume CPU cycles too.

6

I don't believe these temperatures are excessively high. Before adding a fan this summer, I observed temperatures ranging from 60 to 90 degrees Celsius (90 degrees was due to my mistake, as the router was placed near a high-temperature mini PC), and there were no reboots for weeks.

Currently, I've configured the logs to be written to the HDD and will monitor what occurs. Regarding the kernel log, how can I configure it to write to a file instead of RAM?

7

I experience the same problem of reboots.
Have you found what caused your router reboots?

8

or maybe the reason is something else, for example, security
change port for SSH access
if there is luci, then disable login via port 80 and make login only via https by changing port 443 to some other.
block igmp and icmp protocols via firewall menu luci, this way you won't be able to turn off icmp completely, but there will be less flooding, the only thing is that ping won't work.
mtu setup manually
block vulnerable ports
if the settings are more strict, then here is raw_prerouting
for example
iifname "eth0.2" ip protocol igmp drop
same for the exit raw_output
oifname "eth0.2" ip protocol igmp drop
for example, you don't use an iptv provider, then why keep the igmp protocol, block it just in case :slight_smile:
protocols pim, rdp, I don't think you use them, block them too
block these ports 135, 137, 138, 139, 445, 1900, they should not go online :slight_smile:
if there is remote access to your router, then disable it or configure it so that the input is only from a specific IP
I remember a girl turned on VPN on her phone and the router just froze and until you reboot it manually it won't work - this is just one example

9

No, I haven't. And I noticed that decreasing the load eliminates spontaneous reboots completely.