OpenWrt Forum Archive

Topic: Reverse ssh tunneling HOW-TO

The content of this topic has been archived on 23 Mar 2018. There are no obvious gaps in this topic, but there may still be some posts missing at the end.

Hi everyone

Previously I posted about a new device I had purchased:

https://forum.openwrt.org/viewtopic.php?pid=238822

I intend to use the device as a remote serial port server which tunnels back to my home server so that I can access it wherever it goes.



This how-to may be missing some steps and details. Please reply if you can suggest improvements.

Now that I have my little device with it's serial port set up, I need a way to access it remotely.  I wrote a script which starts a SSH reverse tunnel to my server at home to do this.

Let's call this device "probe1".

The plan is to be able to mail the probe to my on-site helper. The helper will plug the device into the wall for power, plug the WAN port into their LAN, and connect the USB serial port to the device I need to work with.

To make sure my helper doesn't get confused and plug into the LAN port by mistake, I've crimped a dummy plug and put it into the LAN jack on my device.

Once the probe is plugged in the customer's LAN, it should automatically get a DHCP address, route, and DNS.

I wrote a script daemon to start and stop the remote ssh tunnel. I called it tunnelhome:

    http://pastebin.com/rczK5XTn

    ./tunnelhome start
    ./tunnelhome stop

Put this file somewhere and make it executable. Then add a line to your /etc/rc.local to start it at boot.

My tunnelhome daemon will detect that the network is up and automatically try to start the ssh reverse tunnel. It's also smart enough to stop when the network has gone away or changed, and to try and restart itself.

The following extra packages are needed to run this script. Be warned that the OpenSSH client requires nearly 2MB of flash:

opkg install openssh-client openssh-client-utils procps-pgrep

We must use OpenSSH client here because Dropbear's dbclient is really dumb and won't die even if the network goes down. dbclient will wait until the OS itself closes the port, and that could take 15-20 minutes.  If you really wanted to use dbclient you can, but you will need to change the script a little and use dropbear below instead of ssk-keygen.

Note that you do not need to create a firewall rule on the probe for this ssh reverse tunnel to work.  As long as the probe can ssh outbound to our home server, we can then ssh back into it.

Configure the variables near the top of tunnelhome as needed for your system. Here we assume our probe's name is "probe1", we are going to use localhost port "22201" on our home server, and the home server is "foo.bar".



On the probe, we must create a private/public key pair. Do the following as root:

    ssh-keygen -t rsa -b 4096
    # This took several minutes on my 400Mhz AR9331 CPU.
    # Save the file in the default; /root/.ssh/id_rsa
    # Do not supply a password.

You should now have a public key:

    ~/.ssh/id_rsa.pub



Now we need to set up the remote home server.

Add a new group named "probes":

    sudo addgroup probes

Add a system user for probe1:

    adduser probe1 --system

Add the probe1 user to the probes group:

    adduser probe1 probes

Now, set up the user's home directory:

    # We will assume that your user account here is a member of the "staff" group, but you could use another group instead.
    sudo mkdir /home/probe1/.ssh
    sudo touch /home/probe1/.ssh/authorized_keys
    sudo chown -R probe1:staff /home/probe1
    sudo chmod -R o-wrx,g-w,g+rx /home/probe1
    sudo chmod -R o-wrx,g-wx /home/probe1/.ssh/authorized_keys
    sudo vi /home/probe1/.ssh/authorized_keys # Import the public key from above into authorized_keys now.

Next, add the following to your sshd_config file on your home server:

    match group probes
            PermitOpen none
            AuthenticationMethods publickey
            AllowTcpForwarding remote
            PermitTunnel yes
            GatewayPorts no
            PermitTTY no
            X11Forwarding no
            AllowAgentForwarding no
            ForceCommand echo nope

    match user probe1
            PermitOpen localhost:22201

For each new probe you add, add the "match user probeX" portion above with a new user and port (probe1 = 22201, probe2 = 22202, ...).

This sshd configuration will allow the remote probe to start the reverse tunnel, but won't allow the account to get a shell, scp/sftp files, or do anything else.

Remember to add probes to your "AllowGroups" statement IF you already use AllowGroups. Do not start using AllowGroups unless you know what it does (you could lock yourself out of your home server).

Restart your home server sshd.  Also, test a regular user to make sure you didn't break anything.

Your probe should be able to start the reverse ssh tunnel now.



To test the ssh reverse tunnel, do this on the probe:

    ssh -i /root/.ssh/id_rsa -N -R 22201:localhost:22 probe1@foo.bar
    # Where foo.bar is your home server.

On your home server, do:

    ssh root@localhost -p 22201
    # And provide your OpenWRT root password, assuming you have not set up key authentication.

If this works, great. Otherwise, better start troubleshooting.  Make sure your public key was imported properly into probe1's authorized_keys file, that file permissions on your .ssh directory and files are good, etc etc; Check your server's syslog auth.log for clues.

Reboot your probe to make sure that the tunnelhome script works at boot time. Check "ps w" output and look for the ssh command. Also, ssh into your probe from your home server to be sure.



Finally, to make connecting to the new probe easy, put the following into your regular home server user's ~/.ssh/config file:

host probe1
        hostname localhost
        user root
        port 22201

Now you don't have to remember which port you assigned to which probe.  Just do "ssh probe1".



If I missed anything in these instructions, let let me know so that I can improve them.

(Last edited by jmomo on 26 Jul 2014, 01:33)

FYI Dropbear updated the -K argument per my feedback which seems to resolve the issue I noted above.  This should make dbclient die faster after a network disruption.

https://github.com/mkj/dropbear/blob/master/CHANGES

2014.64 - Sunday 27 July 2014

- The -K keepalive option now behaves more like OpenSSH's "ServerAliveInterval".
  If no response is received after 3 keepalives then the session is terminated. This
  will close connections faster than waiting for a TCP timeout.

If you use the openssh client, note that it would be a very good idea to add the -o "ExitOnForwardFailure=yes" option. This will make the ssh client exit if the server remote port is already in use. This can happen when you reboot your device. Otherwise, the default ssh client behavior is to just issue a warning message, but the client never disconnects.

Without this option, the following warning is printed, but the client continues to run (this is bad):
Warning: remote port forwarding failed for listen port 22201

With the option, the following error is printed, and the client quits (this is good):
Error: remote port forwarding failed for listen port 22201

The discussion might have continued from here.