I have 22.03.2 running on an R7800 and a Raspberry Pi 4. On both setups the root password keeps changing on its own, seeming at random after a certain number of reboots. LuCI fails the login request with my known root password - I have no idea what openwrt thinks the password is at that point since it's stored as a hash. Fortunately I have ssh configured, which allows me to perform a password-less login with my ssh key, at which time I change the root password back to what it was via passwd. After doing this LuCI accepts the login.
It's getting pretty annoying. Does anyone know what might be happening? When I look at /etc/shadow I see that the entry for root has changed when LuCI fails the login.
Unless you have a file corruption issue, this should not happen and, to my knowledge, has not been reported by other users. Does anyone else have access to your devices? Did you change any of the default firewall settings?
Nobody has access and the routers aren't exposed to the internet yet so no chance of a compromise. Media corruption seems unlikely since its occurring on two different router models, one with internal NAND (r7800) and the other using a USB thumb drive (Raspberry Pi 4).
Here's a list of packages i installed (or were installed as dependencies of the ones I installed), or possibly were updates to pre-installed packages. I generated this list via the script here.
The configuration changes vary between the r7800 and the Pi4, although I started seeing the issue pretty early during my pre-production testing of the two.
I'd take a look at all the packages that have the potential
to modify login related things... for example (not exhaustive):
and probably others.
If you an actually figure out how to repro the problem (i.e. 10 reboots, etc.), you can then start with a fresh install and verify it doesn't happen in that case (I'm pretty sure it'll be fine), then install certain packages that could affect this behavior and continue testing.
Run a logread -e password and look for messages: Fri Jan 6 10:05:13 2023 auth.err passwd: password for root changed by root
You can add -f to follow the buffer.
Better yet, add a syslog destination.
I agree with the others, this is a rather bizarre thng to happen. It have never seen it in well over a decade and dozens of devices and versions of OpenWrt, and I would remember because it is a rather worrying thing to happen. It really has to be due to a rather unconventional package you installed, a script you installed (init.d? rc.local?), or a configuration change you made.
You do not mention your level of knowledge/familarity with OpenWrt or Linux, so maybe you could elaborate on how you do your configuration? Did you follow random instructions from the internet (which one?) or are you doing these changes by yourself?
Can you provoke the issue, is it reproducible? For example, does it happen reliably after half a dozen reboots, or a dozen? If so, I would suggest starting over with a fresh and default install, doing your configuration changes as granular as possible, one by one, and then try to provoke the issue after each step. This way you can find which one of your changes introduced the behaviour.
Finally, what gets my senses tingling is you mentioning
This is not the default and in the same area as the password, you had to work in exactly the area that seems affected now. I'm curious exactly how you did that, which instructions you followed, and which scripts or packages you installed to do that.
I'm a longtime daily Unix and Linux user (20+ years), though I dare not call myself an expert since it takes several lifetimes to achieve that in a system so deep I've also developed Linux kernel drivers though it's been a few years. I did debug a nasty CPU cache corruption issue on Openwrt with a defective R7800 SoC (link) - that router was tossed out.
All of my configuration changes are rather tame and nearly all were done through LuCI so nothing much more than the average seasoned Openwrt user. Nothing exotic for sure.
Not sure why my SSH setup caught your eye. It's pretty standard setup with a public/private key pair in place of using/allowing user/root logins for SSH.
I've removed bash on the routers, as I agree with @psherman that is the most likely culprit. Haven't seen the password change since doing that yesterday. Will report back either way.
Sometimes people come in here with a problem and then it turns out they have been following some random how-to, or installed some random script without exactly knowing what it does. Clearly not the case for you, though, so no offense please.
bash being the cuplrit would be slightly worrying, and would warrant digging a bit deeper, if it's just an incompability somewhere (random thought: does bash "upgrade" password hashes to a method that rest-of-OpenWrt cannot work with anymore?) or an outright bug.
It's not bash apparently - it just occurred again. And more strangely, on both routers at the same time after both were rebooted in tandem by me. Each shows a different hashed password. Both show the same last-updated date stamp (# days since time_t) in field #3 of /etc/shadow and the value indicates today, which means whoever is changing the password is correctly updating this field.
As for the setup, the two routers are connected to each other over one of the R7800's ethernet ports, running a Wireguard connection to facilitate some pre-deployment testing. Both are configured as WAN clients to simulate internet connectivity.
A few years ago I had the same - password changing all by itself.
I went through the WTF stage quite quickly and found my build was actually too big, resulting in the overlay partition being created in tmpfs.
Of course I set the password and played all day with the device,no problems. Then rebooted..... password changed.... well actually no, it had reverted to the password I had set in the build. Took me a while....
All official packages and systems are isolated from the net.
The system log doesn't show a password-change entry when this occurs, so the password isn't being changed via the passwd executable. (I checked the busybox source and the generation of that log message is unconditional).
I've written a script that monitors /etc/shadow and will log a message if a change is detected. If it occurs during the non-boot portion of the power-on session then I can at least try to correlate it to other log messages or something I was doing at the time.
Does the password hash type (the first number between $ ) change?
(Still thinking along the lines that bash, dropbear or something wants to change the hash type. But that should only happen when changing the password)
I'm no expert but one thought I had is to look at file mtime, ctime, atime timestamps of all files in directories that might show application log, installation or config changes to see if there is a correlation. It might point to a process also doing a password change. If nothing correlates close in time, look at files with later change times that might have updated multiple times since the event. Include /tmp in the search.
At least on the R7800 that has a normal rom+overlay approach, you can also more easily monitor for other changed files, as all run-time changes get written to /overlay/upper , so you usually have a rather small number of files that have been modified.
You might look in /overlay/upper to see if any other files there have similar timestamps as /etc/shadow. (Note that it is actually /overlay/upper/etc/shadow that has changed. /rom/etc/shadow should be without passwords unless you have burned something into the firmware)
I can't see to figure out how to quote a reply on this forum, lol.
Hash type doesn't change when the password is updated.
I checked other files in /etc and /overlay/upper/etc and don't see any updated in the same period as shadow, using:
I've also developed Linux kernel drivers though it's been a few years. I did debug a nasty CPU cache corruption issue on Openwrt with a defective R7800 SoC (