Openssh-client stuck on SSH2_MSG_KEX_ECDH_REPLY

Andalibi · September 14, 2023, 3:08pm

Device: TP-Link MR6400 (v5.3)
Version: OpenWRT 22.03.5
Openssh-client Version: 8.9p1-1 (latest)

After installing openssh-client package I am able to connect to some ssh servers but not to the others! So I googled and found there are several ways to manage to connect to the others such as defining KexAlgorithms (either in .ssh/config file or as a flag in command), defining MACs or changing MTU.

The problem is that although these solutions fix the situation, after appending -D flag for dynamic forwarding the ssh process eats up CPU and it's CPU usage soars up to 98% and stays on. I am looking for a solution based on applying changes on ssh servers. I would send ssh logs of both servers which look exactly the same to me, algorithms and ciphers wise.

Server A which connects without needing any configs

root@OpenWrt:~# ssh -p X X@A -v
OpenSSH_8.9p1, OpenSSL 1.1.1v  1 Aug 2023
debug1: Reading configuration data /root/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Connecting to A [A] port X.
debug1: Connection established.
debug1: identity file /root/.ssh/id_rsa type 0
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa_sk type -1
debug1: identity file /root/.ssh/id_ecdsa_sk-cert type -1
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: identity file /root/.ssh/id_ed25519_sk type -1
debug1: identity file /root/.ssh/id_ed25519_sk-cert type -1
debug1: identity file /root/.ssh/id_xmss type -1
debug1: identity file /root/.ssh/id_xmss-cert type -1
debug1: identity file /root/.ssh/id_dsa type -1
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.9
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.2p1 Ubuntu-4ubuntu0.7
debug1: compat_banner: match: OpenSSH_8.2p1 Ubuntu-4ubuntu0.7 pat OpenSSH* compat 0x04000000
debug1: Authenticating to A:X as 'X'
debug1: load_hostkeys: fopen /root/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ssh-ed25519
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host key: ssh-ed25519 SHA256:---
debug1: load_hostkeys: fopen /root/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: Host '[A]:X' is known and matches the ED25519 host key.
debug1: Found key in /root/.ssh/known_hosts:1
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 134217728 blocks
debug1: Will attempt key: /root/.ssh/id_rsa RSA SHA256:---
debug1: Will attempt key: /root/.ssh/id_ecdsa
debug1: Will attempt key: /root/.ssh/id_ecdsa_sk
debug1: Will attempt key: /root/.ssh/id_ed25519
debug1: Will attempt key: /root/.ssh/id_ed25519_sk
debug1: Will attempt key: /root/.ssh/id_xmss
debug1: Will attempt key: /root/.ssh/id_dsa
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,sk-ssh-ed25519@openssh.com,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256@openssh.com>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Offering public key: /root/.ssh/id_rsa RSA SHA256:---
debug1: Server accepts key: /root/.ssh/id_rsa RSA SHA256:---
Authenticated to A ([A]:X) using "publickey".
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: pledge: filesystem
debug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0
debug1: client_input_hostkeys: searching /root/.ssh/known_hosts for [A]:X / (none)
debug1: client_input_hostkeys: searching /root/.ssh/known_hosts2 for [A]:X / (none)
debug1: client_input_hostkeys: hostkeys file /root/.ssh/known_hosts2 does not exist
debug1: Remote: /home/X/.ssh/authorized_keys:2: key options: agent-forwarding port-forwarding pty user-rc x11-forwarding
debug1: Remote: /home/X/.ssh/authorized_keys:2: key options: agent-forwarding port-forwarding pty user-rc x11-forwarding
debug1: client_global_hostkeys_private_confirm: server used untrusted RSA signature algorithm ssh-rsa for key 0, disregarding
debug1: update_known_hosts: known hosts file /root/.ssh/known_hosts2 does not exist

Server B which couldn't connect without configs

root@OpenWrt:~# ssh -p X X@B -v
OpenSSH_8.9p1, OpenSSL 1.1.1v  1 Aug 2023
debug1: Reading configuration data /root/.ssh/config
debug1: /root/.ssh/config line 1: Applying options for B
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Connecting to B [B] port X.
debug1: Connection established.
debug1: identity file /root/.ssh/id_rsa type 0
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa_sk type -1
debug1: identity file /root/.ssh/id_ecdsa_sk-cert type -1
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: identity file /root/.ssh/id_ed25519_sk type -1
debug1: identity file /root/.ssh/id_ed25519_sk-cert type -1
debug1: identity file /root/.ssh/id_xmss type -1
debug1: identity file /root/.ssh/id_xmss-cert type -1
debug1: identity file /root/.ssh/id_dsa type -1
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.9
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.2p1 Ubuntu-4ubuntu0.9
debug1: compat_banner: match: OpenSSH_8.2p1 Ubuntu-4ubuntu0.9 pat OpenSSH* compat 0x04000000
debug1: Authenticating to B:X as 'X'
debug1: load_hostkeys: fopen /root/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ssh-ed25519
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY

lleachii · September 14, 2023, 3:16pm

Welcome to the community!

To be clear, you're seeking a solution to the high CPUs while connected to a remote SSH server - using the local OpenWrt device?
To be clear, you're seeking a solution to fix this on the remote device, correct?
- If so, is the remote device running OpenWrt?

Andalibi · September 14, 2023, 3:51pm

I Would rather this solution.
Remote SSH server is running on an Ubuntu 20 (VPS).

lleachii · September 14, 2023, 4:01pm

What solution?

I asked a question for clarity. So to be clear, you're asking us to assist you with the Ubuntu machine - not the OpenWrt?

(I'm asking because this is an OpenWrt-related forum - and your response isn't clear.)

If you're seeking how to reconfigure OpenSSH on Ubuntu - I would suggest the Ubuntu Forums.

My suggestion: select an algorithm that doesn't use as much CPUs.

vgaetera · September 15, 2023, 12:55am

Consider other tunneling solutions, such as VPN and specifically WireGuard, since it is reliable, easy to setup, provides great performance, and highly compatible, so your port forwarding should not be a problem.

Andalibi · September 15, 2023, 6:03am

Let me explain in other words.

I have two exactly similar remote ubuntu vps, both running ssh server with exactly the same configs. My OpenWRT is able to connect to first one via openssh-client, however, unable to connect to the second one.

What might the problem be? I have sent the logs in the #1 post of this topic.

vgaetera · September 15, 2023, 6:39pm

If you can connect to one server, but not the other, it looks like a problem on the server side, e.g. a firewall rule redirecting SSH to another host, or some difference in the configs, security policies, package versions, build flags, or possibly corrupted keys, but it might also be a client side issue, e.g. perhaps you are using the wrong IP, or the wrong host public key, or the issue is caused by MITM, etc.