Remove Failsafe from WAN / RJ11 Telephone ADSL Port

Site Feedback and Other Questions
1

So I’m minding my own business one day when my ISP sends a new router which I didn't ask for - it just turned up. After a brief inspection I notice it’s made by Huawei and promptly re-boxed it - the cell phone networks are removing Huawei equipment, so why are broadband companies pushing it?

Not long after that I get a letter from my ISP saying that they noticed I hadn’t used the new router and if I didn’t use it then I may lose broadband service... So, I decided to take a look at their router only to find TR-069 and UPnP are turned on by default and I raised these concerns on their forum:

Needless to say I’m a tin foil hat wearing person for refusing to use Huawei equipment and clearly should just plug in their router without a seconds thought. Currently I use OpenWrt for ADSL and I never did get an answer on their forum as to whether I would lose service if I continued to use it. It’s still working - always has.

Anyway some of the replies mentioned that it may be a switch from ADSL to VDSL, so I did some experimenting to find out why they were advising “Please connect your new router without delay to ensure you do not lose your broadband service.” Perhaps it is VDSL rather than ADSL.

At first I tried a Netgear D7000 as this supposedly had ADSL, ADSL2, ADSL2+ and also supported VDSL. Verdict: ADSL2+ is no faster than the current ADSL option in the OpenWrt router that I’m using. Sounds more like it’s capped at the ISP side. So then I started testing out VDSL with the D7000 - nothing, just wouldn’t connect. Okay, so now try the OpenWrt router - same, no connection. Conclusion: VDSL is not enabled on my line.

Then something weird happened.

You see in order to test the routers I had hooked them in to my PC direct -normally I have pfSense sitting in between them, but when you’re testing things you should always remove any other possibilities from any situation. So the network was just literally my PC connected to the OpenWrt router and the RJ11 cable to the phone socket (yes, with a filter). No WiFi running. I’m in the middle of running a speed test and the wesbsite declares it failed to run properly and then the router just restarted all by itself. Weird! However, I didn’t think anything of it as I was mucking about with the settings trying to get VDSL to work. Perhaps it was something I did, oh well never mind. Like I say just my PC, the OpenWrt router, the ADSL connection and no WiFi. Very weird.

The very next day (after I plug OpenWrt back in as the ADSL provider with pfSense getting a feed as “WAN” from OpenWrt) I notice a lot of strange connections appearing in the pfSense logs. They’re all 192.168.1.1:41656 attempting to connect to 192.168.1.255:4919 using UDP. (Yes, you’ve probably guessed where this is going). I don’t use the 192.168.1.1 range for private networking, I use the 10.0.0.1 range which is why it jumped out at me. So what’s going on?

Well after some investigating it turns out that port 4919 is rather special for OpenWrt - it’s a back door! You kids and your new fangled Hex 1337 (Leet) humour! There’s no way that any device should have been attached to the OpenWrt box using the 192 range. The only things connected are an RJ11 cable, one Ethernet as “WAN” to pfSense and the WiFi is switched off. This has to have come from my ISP! Surely not? Tin foil on stand by...

Later on I discover that there is a Failsafe mode which runs only for a short period of time during boot listening on port 4919 which happens to give password-less Root access with Telnet on the 192.168.1.1 range. That explains the sudden reboots for no reason then. So, why is this running on the OpenWrt router on the WAN? You know the RJ11 ADSL telephone connection? Only my ISP could have predicted the next IP address that I would get when the router rebooted as they are dynamically assigned every time you restart the router. It has to be from my ISP!

I get it that you might accidentally lock yourself out of your device and need to use this backdoor, but surely it could be made to run on LAN ports only. By exposing it to the WAN (RJ11 ADSL telephone connection) during boot you are temporarily allowing ISP’s an opportunity for accessing the router with Root privileges. Yikes!

Needless to say I’m not overly impressed and if it hadn’t been for pfSense then I would never have known.

Could you please remove Failsafe from the WAN port, or better yet just get rid of it / have the ability to turn it off without modifying code and compiling yourself. I mean, no password Root Telnet access..! Really???

You could argue they have the right to access equipment connecting to their network, but I doubt they would be best pleased if I took the same approach and accessed their network equipment as Root using my equipment, and besides they clearly tried to go beyond pfSense to access the devices on my network. That’s definitely crossing the line. Fortunately that didn’t happen.

Needless to say I have since flashed the router with a fresh copy of OpenWrt and reset pfSense as a precaution.

Please, please, remove Failsafe from the WAN / RJ11 port - they have no right accessing my equipment. I’m still in contract, but let’s just say I’ll be looking around at other ISP’s when the time comes.

2

Forgive me if I'm missing the obvious in this wall of text, but it might me useful if you could explicitly state the exact device you're accusing of misbehaviour and the OpenWrt version running on it. I'm not even 100% positive if you're talking about vanilla OpenWrt or some vendor firmware with an identity crisis, which still bears markings of OpenWrt despite having very little in common with it.

Also please clarify what exactly you're seeing on which port(s), e.g. the dsl interface in particular.

1 Like
3

Plusnet One Hub, openwrt-21.02.1-lantiq-xrx200-bt_homehub-v5a-squashfs-sysupgrade.bin just installed. I'm just requesting you remove the port 4919 Failsafe is removed from the WAN / RJ11 connection. Thanks.

pfSense blocked 192.168.1.1:41656 to 192.168.1.255:4919 using UDP. I don't use the 192 port range just 10.0.0.1 range which is why it stood out in the logs.

4

The failsafe mode only listens to connection on the ethernet ports, I cannot imagine how could it listen to connections on the ADSL line.

Unless the pfsense box is sitting between the router and the ADSL connection, I do not understand what makes you think that OpenWrt is accepting connections on WAN interface.

1 Like
5

The Plusnet One has OpenWrt which does the ADSL (dial up) and pfSense is the firewall which everything else is attached to. The exact message was:

Feb 13 08:17:17 WAN Block private networks from WAN block 192.168/16 (12004)
192.168.1.1:41656 192.168.1.255:4919 UDP

As far as pfSense is concerned OpenWrt is the WAN.

It’s clearly coming from The Plusnet One running OpenWrt and pfSense isn’t having any of it. Like I say there is no WiFi running, there is an RJ11 cable supplying internet (WAN) and Ethernet going to pfSense, which pfSense is treating as the WAN.

They clearly had a connection established on OpenWrt with the IP 192.168.1.1 which they then tried to connect to pfSense on and failed. The whole network is running on the 10.0.0.1 range (OpenWrt and pfSense) - it must have come from the internet through the RJ11 cable. No other possibility.

To be honest I’ll just modify the source code and recompile. I believe this is the Failsafe code:

6

So, you are connecting the LAN side of your OpenWrt box to your pfSense WAN side and are now wondering why that gives you a regular 192.168.1.1 failsafe there? Well, it's just the LAN side after all.

3 Likes
7

As @sumo commented, the pfsense box is seeing those packets on its WAN interface, that is connected to the LAN interface from the OpenWrt router.

There is nothing coming from the ADSL port, it's all on the ethernet port.

2 Likes
8

It should also be noted that the packets being picked up (i.e. to 192.168.1.255:4919) are broadcast packets to the entire 192.168.1.0/24 subnet (the default for OpenWRT) to notify that a button needs to be pressed on the router if you want to enter failsafe mode. They're notifications, not attempts to connect to anything.

4 Likes
9

Thank you. The OpenWrt router is set to a static 10.0.0.1 IP, with DHCP set to run between 10.0.0.100 to 10.0.0.110. pfSense gets its WAN IP from this range. So how did it manage to get the 192.168.1.1 range? More to the point, nobody is really answering why it randomly rebooted, or why I have never seen this being logged by pfSense before? Also, since it's been flashed it's not doing it any more. I have had this set up for some time now and would have noticed this before.

Anyway, regardless as to the true cause it's not doing it now - let's just hope it remains that way!

10

Because that's what the u-boot environment defaults to before the full firmware (and config files) get loaded.

Routers crash. It happens. But there are many reasons why it can happen and without spending a fair bit of time on diagnostics, various testing steps, and seeing config files and logs it's very difficult to say exactly why the crash occurred.

I can't answer why it's not been logged in pfSense before, but assuming the failsafe code is still in the newly flashed image the router will send out a broadcast packet when rebooting. However, it's completely harmless. It has nothing to do with anything on the internet side of the router.

11

You have some valid points there - it could just be random crashes of the router. However, considering that it hasn’t crashed since it’s been flashed I would rule out hardware, which really just leaves software. Just imagine if Tesla took that attitude... Perhaps it was a buffer overflow with some very unusual circumstances. I get that writing near perfect software is difficult - maybe Rust would help prevent that?

Now that it’s established that it is Failsafe - why is it running every time it starts? Wouldn’t it be best practice to have the router start as normal without Failsafe running and only drop down to Failsafe when it’s unable to start properly? That way the attack surface is significantly reduced.

12

There's simply not enough information to conclude whether the issue was software or hardware related. The fact it hasn't crashed since being flashed doesn't rule out an intermittent hardware fault. But either way, if it's not crashing on a regular basis (or at all anymore) it's probably not worth spending a lot of time trying to diagnose it.

How is the initial boot environment supposed to know whether or not OpenWRT is going to load successfully?

What attack surface do you think there actually is here?

13

So on some MIPS based routers one can use:
cat /sys/kernel/debug/crashlog > last_crash_log

to get some information about a crash after the following warm-reboot, sometimes this works and is actually helpful... also not clear whether that still works on 5.N kernels...

14

Nowadays one would use PSTORE ramoops.

15

That is rather involved and requires to built a new kernel/firmware image. The beauty of the old method was that is basically was available by default. Well, what can you do....

1 Like
16

If you can assign OSI layer 3 IP addresses with Failsafe then there must be a sufficient amount of the embedded Linux OS system running that would allow storage in some form or fashion. I’m pretty sure there are quite a few registers that could be used.

A primitive form of setting a “success” variable initially to false. Upon a successful boot it’s set to true. The next time it starts if success is set to true then normal boot is done, however if it’s set to false then it never completed the boot sequence and it then it runs the Failsafe and tries to continue with boot sequence as it does at the minute.

Surely this has got to be better than exposing port 4919 with Root access Telnet every time Openwrt boots?

17

Have you managed to actually log into the router via telnet on 192.168.1.1 as root without a password? According to the wiki this packet is really just informing you that if you press any button on the router now you will find your self in failsafe mode (this information can be helpful for routers without or broken LEDs that signal the same failsafe is trigger-able right now period to an observer ). And only after the user pressed a button in the right period will the router actually boot into fail safe mode.
Note how pressing a button means you have or easily could have physical possession of the router, at that point it is game over anyway, no?

2 Likes
18

The window in which a button press does boot into failsafe mode is also limited to 2 seconds. And as you say, if an attacker has physical access to the device then you have bigger issues to worry about anyway.

1 Like
19

And as side note to the OP, failsafe requires squashfs, so if you do not want that available on your router do to your threat model, just use don't use squasfs (assuming your router model allows that).

20

If I’m reading this correctly and you’re asking about my personal knowledge and ability then all I can say is I can remember using Pumpkin with TFTP access and DD-WRT as far back as the Windows XP days.

As for squashfs, it’s not exclusive to Openwrt - DD-WRT and some Linux live CD’s use squashfs. I fail to see how this is relevant to Failsafe?

Back on topic, the problem is this: Failsafe and a back door look pretty similar to me. There is no need for Failsafe to run each and every time the router boots. On the other hand a back door would require to be run every time the router boots.

I’ve already pointed out how easy it would be to change this. Personally, I don’t think this is a good idea, but I can see that we are never going to agree on this. Running Failsafe each and every time is a bad design which can lead to unauthorised access.

Matum Stabilis