X86 initiating a clean shutdown randomly (not crashing, no errors found, no kernel panic logs)

Installing and Using OpenWrt
1

I was following this thread but it seems unrelated, so I opened another one: X86 crashing issue

It appears to be a graceful/clean shutdown occurring randomly, so I thought it was the watchdog triggering it, so I placed a nowatchdog flag in grub, problem is still recurring, it seems not to be a hardware fault as based from the logs, it's clean shutdown, with the disk safely unmounting. I have also replaced the power supply, same issue.

I've scoured througout the logs, including dmesg, there are no errors to be found.

Setup:
CPU: Intel i7 2600
Mobo: Intel Q65 with latest bios (A22 revision)
RAM: 32GB RAM
NIC: Realtek RTL8211DN & Intel 82579LM
OpenWRT: OpenWrt 23.05.2 r23630-842932a63d with latest intel microcode

Memtest86+ and CPU Stress test passes fine without crashes

Dmesg log from the computer monitor before the shutdown: https://imgur.com/qc69qQ5

Shutdown occured in: Mon Dec 18 14:34:15 2023 timestamp
Logs:

Mon Dec 18 14:06:09 2023 authpriv.notice sudo:  vincejv : TTY=pts/4 ; PWD=/home/vincejv ; USER=root ; COMMAND=/usr/bin/ssh -i /home/vincejv/.ssh/id_ubuntu_jammy vince@ubuntu.lxc.loc
Mon Dec 18 14:26:35 2023 user.notice nft-qos-monitor: ACTION=update, MACADDR=0e:2f:86:22:30:cb, IPADDR=172.16.0.20, HOSTNAME=OPPO-A52
Mon Dec 18 14:26:36 2023 user.notice nft-qos-dynamic: ACTION=update, MACADDR=0e:2f:86:22:30:cb, IPADDR=172.16.0.20, HOSTNAME=OPPO-A52
Mon Dec 18 14:34:15 2023 daemon.info procd: - shutdown -
Mon Dec 18 14:34:16 2023 daemon.info avahi-daemon[4448]: Interface vethZGsS4H.IPv6 no longer relevant for mDNS.
Mon Dec 18 14:34:16 2023 daemon.info avahi-daemon[4448]: Leaving mDNS multicast group on interface vethZGsS4H.IPv6 with address fe80::fce3:1ff:fe4e:5a80.
Mon Dec 18 14:34:16 2023 daemon.info ipsec: 08[KNL] interface vethZGsS4H deactivated
Mon Dec 18 14:34:16 2023 daemon.info ipsec: 12[KNL] fe80::fce3:1ff:fe4e:5a80 disappeared from vethZGsS4H
Mon Dec 18 14:34:16 2023 kern.info kernel: [28248.446635] lxcbr0: port 1(vethZGsS4H) entered disabled state
Mon Dec 18 14:34:16 2023 kern.info kernel: [28248.453276] device vethZGsS4H left promiscuous mode
Mon Dec 18 14:34:16 2023 kern.info kernel: [28248.458147] lxcbr0: port 1(vethZGsS4H) entered disabled state
Mon Dec 18 14:34:16 2023 daemon.info ipsec: 13[KNL] interface vethZGsS4H deleted
Mon Dec 18 14:34:16 2023 daemon.notice netifd: bridge 'lxcbr0' link is down
Mon Dec 18 14:34:16 2023 daemon.notice netifd: Interface 'lxcbr0' has link connectivity loss
Mon Dec 18 14:34:16 2023 daemon.info avahi-daemon[4448]: Withdrawing address record for fe80::fce3:1ff:fe4e:5a80 on vethZGsS4H.
Mon Dec 18 14:34:16 2023 daemon.info ipsec: 12[NET] using forecast interface br-lan
Mon Dec 18 14:34:16 2023 daemon.info ipsec: 12[CFG] joining forecast multicast groups: 224.0.0.1,224.0.0.22,224.0.0.251,224.0.0.252,239.255.255.250
Mon Dec 18 14:34:16 2023 daemon.err nmbd[6067]: [2023/12/18 14:34:16.633936,  0] ../../source3/nmbd/nmbd.c:59(terminate)
Mon Dec 18 14:34:16 2023 daemon.err nmbd[6067]:   Got SIGTERM: going down...
Mon Dec 18 14:34:19 2023 local6.info clamav-milter[5076]: ClamAV: mi_stop=1
Mon Dec 18 14:34:19 2023 daemon.info ipsec: 00[DMN] SIGTERM received, shutting down
Mon Dec 18 14:34:19 2023 daemon.notice miniupnpd[25908]: shutting down MiniUPnPd
Mon Dec 18 14:34:19 2023 auth.info sshd[4081]: Received signal 15; terminating.
Mon Dec 18 14:34:19 2023 daemon.info vnstatd[4387]: Info: SIGTERM received, exiting.
# --- SYSTEM REBOOT HERE -- #
Mon Dec 18 14:35:47 2023 daemon.crit mdadm[1846]: DeviceDisappeared event detected on md device /dev/md0
Mon Dec 18 14:35:47 2023 daemon.notice procd: /etc/rc.d/S13openssl: Generating engines.cnf
Mon Dec 18 14:35:47 2023 daemon.notice procd: /etc/rc.d/S13openssl: Enabling devcrypto
Mon Dec 18 14:35:47 2023 daemon.notice procd: /etc/rc.d/S13openssl: Enabling afalg
Mon Dec 18 14:35:47 2023 daemon.notice procd: /etc/rc.d/S13openssl: Generating providers.cnf
Mon Dec 18 14:35:47 2023 daemon.notice procd: /etc/rc.d/S13openssl: Enabling legacy
Mon Dec 18 14:35:47 2023 user.notice ipset-create: Creating ipset lists
Mon Dec 18 14:35:47 2023 user.notice dnsmasq: DNS rebinding protection is active, will discard upstream RFC1918 responses!
Mon Dec 18 14:35:47 2023 user.notice dnsmasq: Allowing 127.0.0.0/8 responses
Mon Dec 18 14:35:47 2023 daemon.info dnsmasq[1]: started, version 2.89 cachesize 1000
Mon Dec 18 14:35:47 2023 daemon.info dnsmasq[1]: compile time options: IPv6 GNU-getopt no-DBus UBus no-i18n no-IDN DHCP DHCPv6 no-Lua TFTP conntrack no-ipset nftset auth cryptohash DNSSEC no-ID loop-detect inot>
Mon Dec 18 14:35:47 2023 daemon.info dnsmasq[1]: UBus support enabled: connected to system bus
Mon Dec 18 14:35:47 2023 daemon.info dnsmasq[1]: using only locally-known addresses for test
Mon Dec 18 14:35:47 2023 daemon.info dnsmasq[1]: using only locally-known addresses for onion
Mon Dec 18 14:35:47 2023 daemon.info dnsmasq[1]: using only locally-known addresses for localhost
Mon Dec 18 14:35:47 2023 daemon.info dnsmasq[1]: using only locally-known addresses for local
Mon Dec 18 14:35:47 2023 daemon.info dnsmasq[1]: using only locally-known addresses for invalid
Mon Dec 18 14:35:47 2023 daemon.info dnsmasq[1]: using only locally-known addresses for bind
Mon Dec 18 14:35:47 2023 daemon.info dnsmasq[1]: using only locally-known addresses for lan
Mon Dec 18 14:35:47 2023 daemon.warn dnsmasq[1]: no servers found in /tmp/resolv.conf.d/resolv.conf.auto, will retry
Mon Dec 18 14:35:47 2023 daemon.info dnsmasq[1]: read /etc/hosts - 12 names
Mon Dec 18 14:35:47 2023 daemon.info dnsmasq[1]: read /tmp/hosts/dhcp.cfg01411c - 4 names
Mon Dec 18 14:35:48 2023 user.notice : Added device handler type: bonding
Mon Dec 18 14:35:48 2023 user.notice : Added device handler type: 8021ad
2

what about the shutdown button? maybe is the one failing

3

I definitely have that in mind, but is there anyway to tell from the logs, if the shutdown is caused by the power button?

4

also is there a way to disable the shutdown button in software? it seem possible in desktop OSs like ubuntu.
I'd prefer to shutdown using the shutdown command

Edit: I found the /etc/rc.button files, if i edit these files, maybe it'd disable the hardware buttons
Edit 2: Edited, the rc.button files, and it definitely disabled the button, I replaced it with logger instead of calling the poweroff command, now it's logging when pressed instead of shutting down

5

you can disconnect the power buttoj from the mother board, you just have to use it once you need to turn oj, then open the motherboard and disconnect the shut down button, happened one time to me

1 Like
6

you can check the bios log, if you pc have screen you will know the shut down was caused by software because will say in the screen is shutting down or younwill see the console suddenly becoming active and throwing text like crazy after staying static on

7

Yep this is what I saw here, when I took a video of the screen, it was unmounting hard drives, I don't think a kernel panic or crash would care about unmounting drives?
Video: https://imgur.com/qc69qQ5

8

it would be the start button or is not shutdown but a restart? it is going off ot after going off it goes on?

9

I confirmed that this was the issue, after replacing /etc/rc.button/power with the following script, I no longer have "random" shutdowns, just this random log that the power button was pressed by a ghost :ghost::ghost:

#!/bin/sh

[ "${ACTION}" = "released" ] || exit 0

#exec /sbin/poweroff
exec /usr/bin/logger -t button "Power button was pressed"

return 0

Honestly if OpenWRT had basic troubleshooting tools on what caused the shutdown (power button, crash, sudden powerloss or user initiated command) it'd be a lot easier to troubleshoot, I mean even RouterOS prints out in the logs what caused the last reboot/shutdown

image

1 Like
10

Does RouterOS do it on routers where flash storage wear is a thing ?

11

yes, all their (or at least most) Mikrotik Routers have flash, for how they track and store the shutdown/reboot cause and managing the possible flash wear out, I could only guess since the code is proprietary

Here's a sample log from RouterOS on shutdown/reboot tracking
image

12

you should have marked my answer as solution, i told you it was the power button, and is not a ghost is a bad contact in the motherboard pins where the power button is connected if you try other motherboard you will see it will not shut down, solution? just disconnect the shut down pins and set in the bios wake up after power loss on, so will go on once you pc is plugged on

13

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.