Hi
We have created a VPN connection between our distributed offices so that systems on both our networks can access each other. The topology is given in the image attached
Our OpenVPN Client and servers are running on an OpenWRT based devices. We have been able to setup the OpenVPN connection, as well as set the routes on the network gateways on both sides, so that every system on the client side can ping every system on the server side. i.e 192.168.0.103 can ping 192.168.101.100 and vice versa.
The problem arises when we attempt a connection using any other protocol, for example ssh or even tftp, when accessing any system on the client side LAN through the OpenVPN. The connection is very slow and when examined by tcpdump we see a lot of malformed packets on the tunnel. The following is a the result of a tcpdump on tunnel when on the client side OpenVPN , when we try to ssh from 192.168.101.107 to 192.168.0.103
[ 1056.628225] protocol 0800 is buggy, dev tun0
13:36:48.422935 IP (tos 0x0, ttl 63, id 15744, offset 0, flags [DF], proto TCP (6), length 93, bad cksum 3303 (->ba94)!)
192.168.69.0.16384 > 0.93.61.129.16134: tcp 73 [bad hdr length 0 - too short, < 20] [ 1058.408190] protocol 0800 is buggy, dev tun0
13:36:50.195230 IP (tos 0x0, ttl 63, id 15744, offset 0, flags [DF], proto TCP (6), length 93, bad cksum 3303 (->ba93)!)
192.168.69.0.16384 > 0.93.61.130.16134: tcp 73 [bad hdr length 0 - too short, < 20]
[ 1061.968114] protocol 0800 is buggy, dev tun0
13:36:53.755235 IP (tos 0x0, ttl 63, id 15744, offset 0, flags [DF], proto TCP (6), length 93, bad cksum 3303 (->ba92)!)
192.168.69.0.16384 > 0.93.61.131.16134: tcp 73 [bad hdr length 0 - too short, < 20]
13:37:00.870929 IP (tos 0x0, ttl 63, id 15748, offset 0, flags [DF], proto TCP (6), length 93)
192.168.0.103.ssh > 10.8.0.1.36888: Flags [P.], cksum 0x838d (correct), seq 1:42, ack 1, win 905, options [nop,nop,TS 1
13:37:01.164432 IP (tos 0x0, ttl 62, id 7538, offset 0, flags [DF], proto TCP (6), length 64) 10.8.0.1.36888 > 192.168.0.103.ssh: Flags [.], cksum 0x96f1 (correct), seq 2012, ack 42, win 229, options [nop,nop,TS 0
However the OpenVPN client and servers are able to communicate with each other perfectly, i.e we can ssh from 192.168.101.107 to 192.168.0.106 and we see no issues what so ever and the same is the case for the reverse side. It seems this problem arises when packets from the OpenVPN tunnel are forwarded to the destination system or replies from the destination system are sent to the OpenVPN tunnel
The client side OpenWRT is running on a cavium octeon processor. The version from uname result is as follows
Linux OpenWrt 3.10.87 #16 SMP Wed May 8 15:46:10 PKT 2019 mips64 GNU/Linux
The server side OpenWRT is running on a cavium thunderx processor. The version from uname is as follows
Linux OpenWrt 4.9.0 #13 SMP PREEMPT Fri Jun 22 13:58:29 UTC 2018 aarch64 GNU/Linux
When we replace the client side OpenWRT version 3.10.87 on octeon with the OpenWRT version 4.9.0 on thunderX (All other settings remain the same) we do not see this problem and all communication works perfectly.
Has anyone ever experience any issue like this, and if yes how did you resolve this problem.
We have tried setting the MSS as well as MTUs in the Internet gateways
The OpenVPN versions are
Server
Thu May 16 09:48:44 2019 OpenVPN 2.3.11 aarch64-thunderx-linux-gnu [SSL (OpenSSL)] [LZO] [EPOLL] [MH] [IPv6]
Thu May 16 09:48:44 2019 library versions: OpenSSL 1.0.2j 26 Sep 2016, LZO 2.09
T
Client
Wed May 15 13:12:38 2019 OpenVPN 2.3.11 mips64-octeon-linux-gnu [SSL (OpenSSL)] [LZO] [EPOLL] [MH] [IPv6] built on May 8 2019
Wed May 15 13:12:38 2019 library versions: OpenSSL 1.0.2g 1 Mar 2016, LZO 2.08
The client side and network side /etc/config/openvpn files are below for information
Client side
config openvpn 'myvpn'
option enabled '1'
option dev 'tun'
option proto 'udp'
option verb '3'
option ca '/etc/openvpn/ca.crt'
option cert '/etc/openvpn/my-client.crt'
option key '/etc/openvpn/my-client.key'
option client '1'
option remote_cert_tls 'server'
option remote 'XXXXXXXXXXXX 1194'
option topology 'subnet'
option log '/tmp/openvpn.log'
Server side
config openvpn 'myvpn'
option enabled '1'
option port '1194'
option proto 'udp'
option dev 'tun'
option server '10.8.0.0 255.255.255.0'
option keepalive '10 120'
option ca '/etc/openvpn/ca.crt'
option cert '/etc/openvpn/my-server.crt'
option key '/etc/openvpn/my-server.key'
option dh '/etc/openvpn/dh2048.pem'
option log '/tmp/openvpn.log'
option verb '3'
option route '192.168.0.0 255.255.255.0'
option topology 'subnet'
option mode 'server'
option ifconfig_pool_persist '/etc/openvpn/ipp.txt' # flexo_client,10.8.
option user 'nobody'
option group 'nogroup'
option client_config_dir '/etc/openvpn/ccd'
option client_to_client '1'
option ccd_exclusive
Server side /ccd/my-client
push "route 192.168.101.0 255.255.255.0 10.8.0.1"
iroute 192.168.0.0 255.255.255.0
Thanks and Regards
Mutahir