1 (edited by rwhitby 2008-05-08 03:06:35)

Topic: Linksys locking down serial bootloader console to prevent hacking

I just bought a Linksys WRP400 Wireless G Router with 2 Phone Ports.

I've soldered a serial console on the device, but it looks like Linksys (well, CyberTan actually) have gone out of their way to lock out hackers from this device.

It runs u-boot as the bootloader, but prints "CONSOLE_STATE IS locked" on boot, and does not allow you to interrupt the boot with a keypress.  It also looks like CyberTan are not directing kernel output to the serial port either.

Details are at http://wiki.openwrt.org/OpenWrtDocs/Hardware/Linksys/WRP400 and I will keep that page up to date with what I find out.

All in all, it's a pretty good defence (from Linksys's point of view) against the casual hacker ...

Anyone seen this before?

-- Rod

Re: Linksys locking down serial bootloader console to prevent hacking

rwhitby wrote:

It runs u-boot as the bootloader, but prints "CONSOLE_STATE IS locked" on boot, and does not allow you to interrupt the boot with a keypress.  It also looks like CyberTan are not directing kernel output to the serial port either.
...
Anyone seen this before?

No, I've never seen this before, but maybe the locked state can be avoided  by some GPIO input (i.e. jumper setting)?
Otherwise the bootloader is useless during production - usually you write the bootloader to flash via JTAG and burn the
firmware using the bootloader (much faster than via JTAG)

Disassembling the bootloader code could help, but may prove rather time-consuming unless the bootloader sources are available.

Is there a second serial port available?

3 (edited by rwhitby 2008-07-05 15:24:31)

Re: Linksys locking down serial bootloader console to prevent hacking

There is a serial port, and that is where I got the bootlog from.  But the u-boot console (which is mapped to the serial port) is locked, so it doesn't take keyboard input, and it seems the kernel cmdline is set so that it does *not* output the kernel boot log to the serial console (or anywhere else for that matter).

The way they could have done this, without needing a jumper, is to program an unlocked bootloader with jtag, then flash the firmware (which includes an updated u-boot environment which re-locks the console) using that initial bootloader.  Then after the first reboot the console is permanently locked.

It's u-boot, so the sources must be made available - I have already contacted gpl-violations.org about this.

But even with the source to u-boot, assuming that they have used the technique I have outlined above, and assuming that the web-interface firmware update mechanism does not overwrite the u-boot environment, then it seems they have an effective lockout mechanism which requires jtag to overcome.

Update: Linksys has provided u-boot sources at ftp://ftp.linksys.com/opensourcecode/wrp400/, and this confirms the assumption about a 'console_state' variable used to lock the console.

-- Rod

Re: Linksys locking down serial bootloader console to prevent hacking

He asked if there was a _second_ serial port.

Re: Linksys locking down serial bootloader console to prevent hacking

Wodin wrote:

He asked if there was a _second_ serial port.

My apologies - I misread the question.

No, there is no second serial port as far as I can see.  The connector is a 5 pin connector, 3.3V, NC, RX, TX, GND.

-- Rod

Re: Linksys locking down serial bootloader console to prevent hacking

Quit buying Broadcom (Linksys) based hardware now.

Re: Linksys locking down serial bootloader console to prevent hacking

forum2008 wrote:

Quit buying Broadcom (Linksys) based hardware now.

Um, this board is Marvell-based.

-- Rod

Re: Linksys locking down serial bootloader console to prevent hacking

can you take some images of the pcb please ?

Re: Linksys locking down serial bootloader console to prevent hacking

blogic wrote:

can you take some images of the pcb please ?

I've supplied blogic with PCB photos, and am attempting to add them to the NSLU2 gallery.  If anyone else has an openwrt-related gallery in which I can put the photos, or if I can attach them to this forum somehow, please let me know.

-- Rod

Re: Linksys locking down serial bootloader console to prevent hacking

I've got a little bit further in investigating this device ...

Some interesting CGI commands:

http://192.168.15.1/SysInfo.htm

http://192.168.15.1/CgiCommand.htm?command=Reboot

http://192.168.15.1/CgiCommand.htm?command=TftpOn&user=admin&pass=admin

http://192.168.15.1/CgiCommand.htm?command=TftpOff&user=admin&pass=admin

http://192.168.15.1/CgiCommand.htm?command=FactoryReset

You can see the TftpStatus listed on the SysInfo page, and it changes from Off to On as expected using the commands above.

When the TftpStatus is 'on', you can contact a tftpd server on the device. It responds to Linksys-style authenticated 'put' commands, and starts the power led flashing as if it's erasing the flash, but responds with a 'Password Incorrect' error (where that string comes directly from the tftpd binary that we've extracted from the firmware image). So the password it requires is not the default admin password, nor is it the admin password you can set in the management web pages.

That's as far as I've got.

The /sbin/rc binary has the following /etc/shadow entry in it ...

admin:XkRnx5gdf6RdQ:0:0:99999:7:::

... but I think that's just for the default admin password.  Does anyone know how to check whether 'XkRnx5gdf6RdQ' could be the encoded version of 'admin' ?

-- Rod

Re: Linksys locking down serial bootloader console to prevent hacking

rwhitby wrote:

There is a serial port, and that is where I got the bootlog from.  But the u-boot console (which is mapped to the serial port) is locked, so it doesn't take keyboard input, and it seems the kernel cmdline is set so that it does *not* output the kernel boot log to the serial console (or anywhere else for that matter).

The way they could have done this, without needing a jumper, is to program an unlocked bootloader with jtag, then flash the firmware (which includes an updated u-boot environment which re-locks the console) using that initial bootloader.  Then after the first reboot the console is permanently locked.

My latest theory is that they flash an initial firmware image, and use web commands to enable a tftpd server, which they then use (with a password which is not known to me) that tftpd server to switch the console_state to locked after testing.

-- Rod

Re: Linksys locking down serial bootloader console to prevent hacking

The serial console state of a CYT based ATA device is governed by the CONSOLE_STATE variable. Chances are this CONSOLE_STATE variable on your device is set to locked to prevent anyone to access its serial-console port. If your device is not factory locked to any specific VoSP, you can easily issue a telnet and/or ssh login into the device (if it does have a dropbear support) and use the setenv command to set the CONSOLE_STATE variable to unlocked as seen here so that you can access its serial-console port. Otherwise, you will need to use the ping hack (may require a Web Developer Adds-on for FireFox) approach to change the CONSOLE_STATE variable to unlocked.

Please let us know how this goes. Good luck.

Mazi

Re: Linksys locking down serial bootloader console to prevent hacking

mazilo wrote:

The serial console state of a CYT based ATA device is governed by the CONSOLE_STATE variable. Chances are this CONSOLE_STATE variable on your device is set to locked to prevent anyone to access its serial-console port. If your device is not factory locked to any specific VoSP, you can easily issue a telnet and/or ssh login into the device (if it does have a dropbear support) and use the setenv command to set the CONSOLE_STATE variable to unlocked as seen here so that you can access its serial-console port. Otherwise, you will need to use the ping hack (may require a Web Developer Adds-on for FireFox) approach to change the CONSOLE_STATE variable to unlocked.

Thanks for your suggestions - I had already determined that the device has no telnetd or sshd servers by examining the rootfs in the firmware image.  An nmap also confirms this.

Trying to ping anything other than a plain ip address or fqdn results in a "Invalid IP Address or Domain Name" error from the web interface.  I'm investigating how to get around the javascript validation code and talk to the apply.cgi script directly, but I'm not hopeful because if they've gone to the trouble to lock u-boot I'm assuming they've plugged the ping hack a long time ago.

-- Rod

Re: Linksys locking down serial bootloader console to prevent hacking

rwhitby wrote:

I'm investigating how to get around the javascript validation code and talk to the apply.cgi script directly, but I'm not hopeful because if they've gone to the trouble to lock u-boot I'm assuming they've plugged the ping hack a long time ago.

Honestly, I don't know; however, some Linksys CYT based ATA devices still support the ping hack while their serial-console ports are locked. If Linksys had released some new/older versions of firmware, you may want to give that a try to see if any new/old firmware has a support for such a ping hack.

Also, you may want to consider to mod the firmware of any Linksys CYT based ATA devices that has a support for the ping hack (or dropbear) and flash it to your device so that you can change its CONSOLE_STATE variable. Beware that this approach may brick your device. So, do this on your own.

Mazi

Re: Linksys locking down serial bootloader console to prevent hacking

mazilo wrote:
rwhitby wrote:

I'm investigating how to get around the javascript validation code and talk to the apply.cgi script directly, but I'm not hopeful because if they've gone to the trouble to lock u-boot I'm assuming they've plugged the ping hack a long time ago.

Honestly, I don't know; however, some Linksys CYT based ATA devices still support the ping hack while their serial-console ports are locked. If Linksys had released some new/older versions of firmware, you may want to give that a try to see if any new/old firmware has a support for such a ping hack.

Also, you may want to consider to mod the firmware of any Linksys CYT based ATA devices that has a support for the ping hack (or dropbear) and flash it to your device so that you can change its CONSOLE_STATE variable. Beware that this approach may brick your device. So, do this on your own.

There is no newer or older firmware for this device.  In fact, Linksys have not officially released any firmware for this device - the image I have (which I have not tested, since I have no known good recovery mechanism if it doesn't work) is from .cz

Usually, bricking a device is easy to fix using the serial console.  In this case, since u-boot is locked, that risk is too high at the moment.  When all other alternatives and avenues of investigation are done, I may consider adding JTAG to the device and changing the console_state variable that way.

-- Rod

Re: Linksys locking down serial bootloader console to prevent hacking

rwhitby wrote:
rwhitby wrote:

There is a serial port, and that is where I got the bootlog from.  But the u-boot console (which is mapped to the serial port) is locked, so it doesn't take keyboard input, and it seems the kernel cmdline is set so that it does *not* output the kernel boot log to the serial console (or anywhere else for that matter).

The way they could have done this, without needing a jumper, is to program an unlocked bootloader with jtag, then flash the firmware (which includes an updated u-boot environment which re-locks the console) using that initial bootloader.  Then after the first reboot the console is permanently locked.

My latest theory is that they flash an initial firmware image, and use web commands to enable a tftpd server, which they then use (with a password which is not known to me) that tftpd server to switch the console_state to locked after testing.

Why couldn't they just build the firmware to, at first boot, run a verification on itself and then toggle console_state? Then just do an initial test boot after the flash...

Re: Linksys locking down serial bootloader console to prevent hacking

Luke-Jr wrote:
rwhitby wrote:
rwhitby wrote:

There is a serial port, and that is where I got the bootlog from.  But the u-boot console (which is mapped to the serial port) is locked, so it doesn't take keyboard input, and it seems the kernel cmdline is set so that it does *not* output the kernel boot log to the serial console (or anywhere else for that matter).

The way they could have done this, without needing a jumper, is to program an unlocked bootloader with jtag, then flash the firmware (which includes an updated u-boot environment which re-locks the console) using that initial bootloader.  Then after the first reboot the console is permanently locked.

My latest theory is that they flash an initial firmware image, and use web commands to enable a tftpd server, which they then use (with a password which is not known to me) that tftpd server to switch the console_state to locked after testing.

Why couldn't they just build the firmware to, at first boot, run a verification on itself and then toggle console_state? Then just do an initial test boot after the flash...

There seems to be quite a bit of instance-specific information which is programmed into this device, and which could not be determined by a single firmware flash.  So I think it's quite likely that they need to program this stuff individually after the first boot anyway, and they wouldn't put this functionality in the tftpd server in the userspace if they weren't going to use it.

If something goes wrong in the factory during these configuration steps, you would want the console unlocked so you can debug and correct the manufacturing process failure.

These devices are also intended for secure provisioning by Voice Providers, so one would expect there to be a separate configuration step where they could program their own ATA configuration into the device.

-- Rod

Re: Linksys locking down serial bootloader console to prevent hacking

OK, I've found the secret incantation to unlock the console.

0) Solder in a serial console.
1) Boot the device to userspace as normal.
2) Access the following secret URL:
    http://192.168.15.1/CgiCommand.htm?command=TftpOn&user=admin&pass=admin
    (replace 192.168.15.1 with the IP address of your wrp400, and use the correct admin password if you've changed it from the default)
3) Reboot the device with a serial console attached.
4) Note that the u-boot console is now unlocked and you can hit Ctrl-C on the serial console during the 1 second u-boot autoboot delay.

You now have a normal u-boot console.

Hitting the matching "command=TftpOff" URL in userspace will lock the console again.

Note that the reset button has no effect on the console_state variable.

-- Rod

Re: Linksys locking down serial bootloader console to prevent hacking

blogic wrote:

can you take some images of the pcb please ?

http://www.nslu2-linux.org/wiki/WRP400/PhotosOfTheInternals

-- Rod

Re: Linksys locking down serial bootloader console to prevent hacking

rwhitby wrote:

The /sbin/rc binary has the following /etc/shadow entry in it ...

admin:XkRnx5gdf6RdQ:0:0:99999:7:::

... but I think that's just for the default admin password.  Does anyone know how to check whether 'XkRnx5gdf6RdQ' could be the encoded version of 'admin' ?

$ python -c 'import crypt; print crypt.crypt("admin", "Xk")'
XkRnx5gdf6RdQ

Re: Linksys locking down serial bootloader console to prevent hacking

rwhitby wrote:

Trying to ping anything other than a plain ip address or fqdn results in a "Invalid IP Address or Domain Name" error from the web interface.  I'm investigating how to get around the javascript validation code and talk to the apply.cgi script directly, but I'm not hopeful because if they've gone to the trouble to lock u-boot I'm assuming they've plugged the ping hack a long time ago.

If the page will work without any Javascript, just turn it off in your browser.

e.g. in Firefox, just go to the "Content" tab in the options and uncheck "Enable JavaScript".  Firefox does not require a page reload for that setting to take effect.  In the past when I've tried with Internet Explorer it has required a page reload.  Don't know about other browsers.

If completely disabling JavaScript breaks the page, you can just save the page to a file and edit it to make sure the form action is correct (i.e. an absolute URL instead of a relative one) and get rid of any Javascript you don't want, or put "return;" statements in as appropriate.  Then load up the page and submit the form as usual.

Re: Linksys locking down serial bootloader console to prevent hacking

Wodin wrote:
rwhitby wrote:

The /sbin/rc binary has the following /etc/shadow entry in it ...

admin:XkRnx5gdf6RdQ:0:0:99999:7:::

... but I think that's just for the default admin password.  Does anyone know how to check whether 'XkRnx5gdf6RdQ' could be the encoded version of 'admin' ?

$ python -c 'import crypt; print crypt.crypt("admin", "Xk")'
XkRnx5gdf6RdQ

Do you have one that goes in the opposite direction? ;-)

In the u-boot environment variables are the following:

shadow_data=admin:YhCA9pXTYavMg:0:0:99999:7:::
admin_pwd=ABWPOsqBBMagI

-- Rod

Re: Linksys locking down serial bootloader console to prevent hacking

Try "crack" or "john the ripper".

Re: Linksys locking down serial bootloader console to prevent hacking

After using the secret CGI commands to get into the bootloader, the next challenge is to get into userspace on this device.

It seems there is a recovery image, but no obvious means to get into it.

However, looking at the u-boot source code provided by Linksys (due to my reporting the lack of such to Wolfgang Denk, the U-Boot copyright holder, and his subsequent effort to get Linksys to put up the sources), there is some code in common/cmd_bootm.c which deals with the boot_image environment variable.

The code there checks for the existence of a valid "full_image" (the normal kernel and rootfs) and boots that if it finds one.

If it doesn't find one, then it boots a "half_image" (which seems to be a recovery filesystem).

Luckily, there is some test code left over in that function, which examines an "owen_15_test" environment variable, and if this variable is set to "h" then it unconditionally boots the half_image.

Booting the half_image gets you a busybox shell, with full networking available, including the 'wget' command :-)

-- Rod

Re: Linksys locking down serial bootloader console to prevent hacking

OK, here's how to use the recovery tftpd server, once you have a serial port attached and have managed to unlock the console:

At the U-Boot console, type:

  setenv shadow_data admin:XkRnx5gdf6RdQ:0:0:99999:7:::
  setenv owen_15_test h
  boot

This will boot a recovery shell, with the tftpd server running, with an admin password set to 'admin'.

You can then use your favourite linksys-tftp client to upload a new cybertan_half_bin or cybertan_rom_bin image (see http://www.nslu2-linux.org/wiki/WRP400/EncapsulatedFirmware for details of what these are).  I expect you can also upload a uboot.bin file, but I'm not going to try that unless there is no other way to continue the investigation.

-- Rod