OpenVPN: Site-to-Site VPN, Routing Issue

Hello community,

recently I've been trying to achieve a "Site-to-Site VPN". Unfortunately I am unable to figure out, how exactly or rather why exactly it fails for me.
The issue is, that the routing from the VPN clients LAN to the VPN server, works just fine, but not vice versa. I am able to reach everything within the VPN servers LAN from my VPN client, but from my VPN server, I am only able to ping the client and not reach its local network.
My whole configuration is a bit complicated, so I am trying to point out only the important things.
VPN subnet: 10.6.5.0/24
VPN server address (within the VPN range): 10.6.5.1
VPN client address (assigned with static openvpn rules): 10.6.5.2
VPN server LAN: 10.1.1.0/24
VPN client LAN: 10.10.60.0/24
VPN server domain: dmz
VPN client domain: rw

Following the OpenVPN server configuration:

verb 3
user nobody
group nogroup
dev tun0
port 1194
proto tcp4
server 10.6.5.0 255.255.255.0
topology subnet
# client-to-client
keepalive 10 120
persist-tun
persist-key
ifconfig-pool-persist static.ips
push "dhcp-option DNS 10.6.5.1"
push "dhcp-option DOMAIN dmz"
push "dhcp-option DOMAIN lan"
push "dhcp-option DOMAIN home"
push "dhcp-option DOMAIN servers.local"
push "dhcp-option DOMAIN lit.int"
push "dhcp-option DOMAIN rw"
route 10.10.60.0 255.255.255.0
push "route 10.6.5.0 255.255.255.0"
push "route 10.1.1.0 255.255.255.0"
push "route 10.10.10.0 255.255.255.0"
push "route 10.10.20.0 255.255.255.0"
push "route 10.10.30.0 255.255.255.0"
push "route 10.10.40.0 255.255.255.0"
push "route 10.10.50.0 255.255.255.0"

( I tried with both client-to-client and without)

ccd file for rw:

ifconfig-push 10.6.5.2 10.6.5.1
iroute 10.10.60.0 255.255.255.0
push-remove redirect-gateway

static.ips file:

rw,10.6.5.2

client config:

verb 3
dev tun
nobind
client
remote <REMOVED> 1194 tcp
auth-nocache
remote-cert-tls server

iptables on the OpenVPN server (I don't use uci for firewalling, firewall service is disabled):

# Generated by iptables-save v1.6.2 on Sun Oct  6 14:40:43 2019
*nat
:PREROUTING ACCEPT [3:1502]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
-A POSTROUTING -o pppoe-wan -j MASQUERADE
COMMIT
# Completed on Sun Oct  6 14:40:43 2019
# Generated by iptables-save v1.6.2 on Sun Oct  6 14:40:43 2019
*mangle
:PREROUTING ACCEPT [4628349:1913725516]
:INPUT ACCEPT [175602:19504230]
:FORWARD ACCEPT [4451984:1891851268]
:OUTPUT ACCEPT [140804:20755753]
:POSTROUTING ACCEPT [4587102:1911798836]
-A FORWARD -o pppoe-wan -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu
COMMIT
# Completed on Sun Oct  6 14:40:43 2019
# Generated by iptables-save v1.6.2 on Sun Oct  6 14:40:43 2019
*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [20737:3822842]
:chn_reject - [0:0]
-A INPUT -p tcp -m tcp --dport 1194 -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -i br-lan -j ACCEPT
-A INPUT -s 10.1.1.2/32 -j ACCEPT
-A INPUT -p icmp -m icmp --icmp-type 8 -j ACCEPT
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p tcp -m tcp --dport 1905 -j ACCEPT
-A INPUT -i tun0 -j ACCEPT
-A INPUT -j LOG --log-prefix "iptables_INPUT_denied: " --log-level 7
-A INPUT -j chn_reject
-A FORWARD -p icmp -m icmp --icmp-type 8 -j ACCEPT
-A FORWARD -i tun0 -o br-lan -j ACCEPT
-A FORWARD -o tun0 -j ACCEPT
-A FORWARD -i br-lan -o pppoe-wan -j ACCEPT
-A FORWARD -i pppoe-wan -o br-lan -m conntrack --ctstate RELATED,ESTABLISHED,DNAT -j ACCEPT
-A FORWARD -j LOG --log-prefix "iptables_FORWARD_denied: " --log-level 7
-A FORWARD -j chn_reject
-A chn_reject -p tcp -j REJECT --reject-with tcp-reset
-A chn_reject -j REJECT --reject-with icmp-port-unreachable
COMMIT
# Completed on Sun Oct  6 14:40:43 2019

Routes and ip configuration on the OpenVPN server (the bold route gets added from OpenVPN):

root@OpenWrt.dmz:/etc/openvpn$ route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         REMOVED  0.0.0.0         UG    0      0        0 pppoe-wan
10.1.1.0        *               255.255.255.0   U     0      0        0 br-lan
10.6.5.0        *               255.255.255.0   U     0      0        0 tun0
10.10.10.0      openwrt-lan.dmz 255.255.255.0   UG    0      0        0 br-lan
10.10.20.0      openwrt-lan.dmz 255.255.255.0   UG    0      0        0 br-lan
10.10.30.0      openwrt-lan.dmz 255.255.255.0   UG    0      0        0 br-lan
10.10.40.0      10.1.1.4        255.255.255.0   UG    0      0        0 br-lan
**10.10.60.0      10.6.5.2        255.255.255.0   UG    0      0        0 tun0**
REMOVED   *               255.255.255.255 UH    0      0        0 pppoe-wan
root@OpenWrt.dmz:/etc/openvpn$ ip a s
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 532
    link/ether 24:f5:a2:c0:4b:c0 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::26f5:a2ff:fec0:4bc0/64 scope link 
       valid_lft forever preferred_lft forever
3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 532
    link/ether 26:f5:a2:c0:4b:c0 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::24f5:a2ff:fec0:4bc0/64 scope link 
       valid_lft forever preferred_lft forever
5: wlan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether 24:f5:a2:c0:4b:c2 brd ff:ff:ff:ff:ff:ff
6: wlan1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether 24:f5:a2:c0:4b:c1 brd ff:ff:ff:ff:ff:ff
7: mlan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether 24:f5:a2:c0:4b:c3 brd ff:ff:ff:ff:ff:ff
8: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
    link/ether 26:f5:a2:c0:4b:c0 brd ff:ff:ff:ff:ff:ff
    inet 10.1.1.1/24 brd 10.1.1.255 scope global br-lan
       valid_lft forever preferred_lft forever
9: eth0.1@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-lan state UP qlen 1000
    link/ether 26:f5:a2:c0:4b:c0 brd ff:ff:ff:ff:ff:ff
10: eth1.2@eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
    link/ether 24:f5:a2:c0:4b:c0 brd ff:ff:ff:ff:ff:ff
11: pppoe-wan: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1492 qdisc fq_codel state UNKNOWN qlen 3
    link/ppp 
    inet <REMOVED> peer <REMOVED>/32 scope global pppoe-wan
       valid_lft forever preferred_lft forever
27: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN qlen 100
    link/[65534] 
    inet 10.6.5.1/24 brd 10.6.5.255 scope global tun0
       valid_lft forever preferred_lft forever
root@OpenWrt.dmz:/etc/openvpn$ 

iptables on the VPN client (same as for the server, I don't use uci, firewall service disabled):

root@OpenWrt:~# iptables-save 
# Generated by iptables-save v1.6.2 on Sun Oct  6 14:43:19 2019
*nat
:PREROUTING ACCEPT [317:72676]
:INPUT ACCEPT [33:2260]
:OUTPUT ACCEPT [82:5499]
:POSTROUTING ACCEPT [14:938]
-A POSTROUTING -o eth0.2 -j MASQUERADE
-A POSTROUTING -o tun+ -j MASQUERADE
-A POSTROUTING -s 10.10.60.0/24 -j MASQUERADE
COMMIT
# Completed on Sun Oct  6 14:43:19 2019
# Generated by iptables-save v1.6.2 on Sun Oct  6 14:43:19 2019
*mangle
:PREROUTING ACCEPT [2260345:2676489596]
:INPUT ACCEPT [14088:2021702]
:FORWARD ACCEPT [2244017:2673937172]
:OUTPUT ACCEPT [12753:2962316]
:POSTROUTING ACCEPT [2256770:2676899488]
-A FORWARD -o eth0.2 -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu
COMMIT
# Completed on Sun Oct  6 14:43:19 2019
# Generated by iptables-save v1.6.2 on Sun Oct  6 14:43:19 2019
*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [12762:2963604]
:chn_reject - [0:0]
-A INPUT -p tcp -m tcp --dport 123
-A INPUT -d 224.0.0.0/4 -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -i br-lan -j ACCEPT
-A INPUT -p icmp -m icmp --icmp-type 8 -j ACCEPT
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A INPUT -i tun+ -m comment --comment "Accept all traffic on all tun interfaces" -j ACCEPT
-A INPUT -i eth0.2 -p udp -m conntrack --ctstate RELATED,ESTABLISHED -m udp --sport 53 -j ACCEPT
-A INPUT -i eth0.2 -p tcp -m conntrack --ctstate RELATED,ESTABLISHED -m tcp --sport 80 -j ACCEPT
-A INPUT -i eth0.2 -p tcp -m conntrack --ctstate RELATED,ESTABLISHED -m tcp --sport 443 -j ACCEPT
-A INPUT -j LOG --log-prefix "iptables_INPUT_denied: " --log-level 7
-A INPUT -j chn_reject
-A FORWARD -p icmp -m icmp --icmp-type 8 -j ACCEPT
-A FORWARD -i br-lan -o eth0.2 -j ACCEPT
-A FORWARD -i eth0.2 -o br-lan -m conntrack --ctstate RELATED,ESTABLISHED,DNAT -j ACCEPT
-A FORWARD -i tun+ -j ACCEPT
-A FORWARD -i eth0.2 -o tun+ -j ACCEPT
-A FORWARD -i br-lan -o tun+ -j ACCEPT
-A FORWARD -j ACCEPT
-A FORWARD -j LOG --log-prefix "iptables_FORWARD_denied: " --log-level 7
-A FORWARD -j chn_reject
-A chn_reject -p tcp -j REJECT --reject-with tcp-reset
-A chn_reject -j REJECT --reject-with icmp-port-unreachable
COMMIT
# Completed on Sun Oct  6 14:43:19 2019

Route and ip configuration from the VPN client:

root@OpenWrt:~# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         192.168.178.1   0.0.0.0         UG    0      0        0 eth0.2
10.1.1.0        10.6.5.1        255.255.255.0   UG    0      0        0 tun0
10.6.5.0        10.6.5.1        255.255.255.0   UG    0      0        0 tun0
10.6.5.0        *               255.255.255.0   U     0      0        0 tun0
10.10.10.0      10.6.5.1        255.255.255.0   UG    0      0        0 tun0
10.10.20.0      10.6.5.1        255.255.255.0   UG    0      0        0 tun0
10.10.30.0      10.6.5.1        255.255.255.0   UG    0      0        0 tun0
10.10.40.0      10.6.5.1        255.255.255.0   UG    0      0        0 tun0
10.10.50.0      10.6.5.1        255.255.255.0   UG    0      0        0 tun0
10.10.60.0      *               255.255.255.0   U     0      0        0 br-lan
192.168.178.0   *               255.255.255.0   U     0      0        0 eth0.2
root@OpenWrt:~# ip a s
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP qlen 1000
    link/ether b0:be:76:e9:82:66 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::b2be:76ff:fee9:8266/64 scope link 
       valid_lft forever preferred_lft forever
30: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
    link/ether b0:be:76:e9:82:66 brd ff:ff:ff:ff:ff:ff
    inet 10.10.60.1/24 brd 10.10.60.255 scope global br-lan
       valid_lft forever preferred_lft forever
31: eth0.1@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-lan state UP qlen 1000
    link/ether b0:be:76:e9:82:66 brd ff:ff:ff:ff:ff:ff
32: eth0.2@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
    link/ether b0:be:76:e9:82:66 brd ff:ff:ff:ff:ff:ff
    inet 192.168.178.2/24 brd 192.168.178.255 scope global eth0.2
       valid_lft forever preferred_lft forever
33: wlan1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-lan state UP qlen 1000
    link/ether b0:be:76:e9:82:66 brd ff:ff:ff:ff:ff:ff
34: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-lan state UP qlen 1000
    link/ether b0:be:76:e9:82:65 brd ff:ff:ff:ff:ff:ff
35: wlan0-1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-lan state UP qlen 1000
    link/ether b2:be:76:e9:82:65 brd ff:ff:ff:ff:ff:ff
36: wlan1.sta1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-lan state UNKNOWN qlen 1000
    link/ether b0:be:76:e9:82:66 brd ff:ff:ff:ff:ff:ff
118: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN qlen 100
    link/[65534] 
    inet 10.6.5.2/24 brd 10.6.5.255 scope global tun0
       valid_lft forever preferred_lft forever
root@OpenWrt:~# 

OpenVPN server log during startup:

un Oct  6 14:46:51 2019 daemon.notice openvpn(server)[18386]: OpenVPN 2.4.5 arm-openwrt-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [MH/PKTINFO] [AEAD]
Sun Oct  6 14:46:51 2019 daemon.notice openvpn(server)[18386]: library versions: OpenSSL 1.0.2r  26 Feb 2019, LZO 2.10
Sun Oct  6 14:46:51 2019 daemon.notice openvpn(server)[18386]: Diffie-Hellman initialized with 2048 bit key
Sun Oct  6 14:46:51 2019 daemon.notice openvpn(server)[18386]: Outgoing Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
Sun Oct  6 14:46:51 2019 daemon.notice openvpn(server)[18386]: Outgoing Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication
Sun Oct  6 14:46:51 2019 daemon.notice openvpn(server)[18386]: Incoming Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
Sun Oct  6 14:46:51 2019 daemon.notice openvpn(server)[18386]: Incoming Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication
Sun Oct  6 14:46:51 2019 daemon.notice openvpn(server)[18386]: TUN/TAP device tun0 opened
Sun Oct  6 14:46:51 2019 daemon.notice openvpn(server)[18386]: TUN/TAP TX queue length set to 100
Sun Oct  6 14:46:51 2019 daemon.notice openvpn(server)[18386]: do_ifconfig, tt->did_ifconfig_ipv6_setup=0
Sun Oct  6 14:46:51 2019 daemon.notice openvpn(server)[18386]: /sbin/ifconfig tun0 10.6.5.1 netmask 255.255.255.0 mtu 1500 broadcast 10.6.5.255
Sun Oct  6 14:46:51 2019 daemon.notice openvpn(server)[18386]: /sbin/route add -net 10.10.60.0 netmask 255.255.255.0 gw 10.6.5.2
Sun Oct  6 14:46:51 2019 daemon.notice openvpn(server)[18386]: Socket Buffers: R=[87380->87380] S=[16384->16384]
Sun Oct  6 14:46:51 2019 daemon.notice openvpn(server)[18386]: Listening for incoming TCP connection on [AF_INET][undef]:1194
Sun Oct  6 14:46:51 2019 daemon.notice openvpn(server)[18386]: TCPv4_SERVER link local (bound): [AF_INET][undef]:1194
Sun Oct  6 14:46:51 2019 daemon.notice openvpn(server)[18386]: TCPv4_SERVER link remote: [AF_UNSPEC]
Sun Oct  6 14:46:51 2019 daemon.notice openvpn(server)[18386]: GID set to nogroup
Sun Oct  6 14:46:51 2019 daemon.notice openvpn(server)[18386]: UID set to nobody
Sun Oct  6 14:46:51 2019 daemon.notice openvpn(server)[18386]: MULTI: multi_init called, r=256 v=256
Sun Oct  6 14:46:51 2019 daemon.notice openvpn(server)[18386]: IFCONFIG POOL: base=10.6.5.2 size=252, ipv6=0
Sun Oct  6 14:46:51 2019 daemon.notice openvpn(server)[18386]: ifconfig_pool_read(), in='rw,10.6.5.2', TODO: IPv6
Sun Oct  6 14:46:51 2019 daemon.notice openvpn(server)[18386]: succeeded -> ifconfig_pool_set()
Sun Oct  6 14:46:51 2019 daemon.notice openvpn(server)[18386]: IFCONFIG POOL LIST
Sun Oct  6 14:46:51 2019 daemon.notice openvpn(server)[18386]: rw,10.6.5.2
Sun Oct  6 14:46:51 2019 daemon.notice openvpn(server)[18386]: MULTI: TCP INIT maxclients=1024 maxevents=1028
Sun Oct  6 14:46:51 2019 daemon.notice openvpn(server)[18386]: Initialization Sequence Completed

VPN Server log upon connecting with the client (rw):

Sun Oct  6 14:48:18 2019 daemon.notice openvpn(server)[18386]: TCP connection established with [AF_INET]<REMOVED>:56580
Sun Oct  6 14:48:19 2019 daemon.notice openvpn(server)[18386]: <REMOVED>:56580 TLS: Initial packet from [AF_INET]<REMOVED>:56580, sid=3f0db01e b40e8bbc
Sun Oct  6 14:48:20 2019 daemon.notice openvpn(server)[18386]: <REMOVED>:56580 VERIFY OK: depth=1, CN=my.server.name
Sun Oct  6 14:48:20 2019 daemon.notice openvpn(server)[18386]: <REMOVED>:56580 VERIFY OK: depth=0, CN=rw
Sun Oct  6 14:48:20 2019 daemon.notice openvpn(server)[18386]: <REMOVED>:56580 peer info: IV_VER=2.4.5
Sun Oct  6 14:48:20 2019 daemon.notice openvpn(server)[18386]: <REMOVED>:56580 peer info: IV_PLAT=linux
Sun Oct  6 14:48:20 2019 daemon.notice openvpn(server)[18386]: <REMOVED>:56580 peer info: IV_PROTO=2
Sun Oct  6 14:48:20 2019 daemon.notice openvpn(server)[18386]: <REMOVED>:56580 peer info: IV_NCP=2
Sun Oct  6 14:48:20 2019 daemon.notice openvpn(server)[18386]: <REMOVED>:56580 peer info: IV_LZ4=1
Sun Oct  6 14:48:20 2019 daemon.notice openvpn(server)[18386]: <REMOVED>:56580 peer info: IV_LZ4v2=1
Sun Oct  6 14:48:20 2019 daemon.notice openvpn(server)[18386]: <REMOVED>:56580 peer info: IV_LZO=1
Sun Oct  6 14:48:20 2019 daemon.notice openvpn(server)[18386]: <REMOVED>:56580 peer info: IV_COMP_STUB=1
Sun Oct  6 14:48:20 2019 daemon.notice openvpn(server)[18386]: <REMOVED>:56580 peer info: IV_COMP_STUBv2=1
Sun Oct  6 14:48:20 2019 daemon.notice openvpn(server)[18386]: <REMOVED>:56580 peer info: IV_TCPNL=1
Sun Oct  6 14:48:20 2019 daemon.notice openvpn(server)[18386]: <REMOVED>:56580 Control Channel: TLSv1.2, cipher TLSv1/SSLv3 ECDHE-RSA-AES256-GCM-SHA384, 2048 bit RSA
Sun Oct  6 14:48:20 2019 daemon.notice openvpn(server)[18386]: <REMOVED>:56580 [rw] Peer Connection Initiated with [AF_INET]<REMOVED>:56580
Sun Oct  6 14:48:20 2019 daemon.notice openvpn(server)[18386]: rw/<REMOVED>:56580 MULTI_sva: pool returned IPv4=10.6.5.2, IPv6=(Not enabled)
Sun Oct  6 14:48:20 2019 daemon.notice openvpn(server)[18386]: rw/<REMOVED>:56580 MULTI: Learn: 10.6.5.2 -> rw/<REMOVED>:56580
Sun Oct  6 14:48:20 2019 daemon.notice openvpn(server)[18386]: rw/<REMOVED>:56580 MULTI: primary virtual IP for rw/<REMOVED>:56580: 10.6.5.2
Sun Oct  6 14:48:21 2019 daemon.notice openvpn(server)[18386]: rw/<REMOVED>:56580 PUSH: Received control message: 'PUSH_REQUEST'
Sun Oct  6 14:48:21 2019 daemon.notice openvpn(server)[18386]: rw/<REMOVED>:56580 SENT CONTROL [rw]: 'PUSH_REPLY,dhcp-option DNS 10.6.5.1,dhcp-option DOMAIN dmz,dhcp-option DOMAIN lan,dhcp-option DOMAIN home,dhcp-option DOMAIN servers.local,dhcp-option DOMAIN lit.int,dhcp-option DOMAIN rw,route 10.6.5.0 255.255.255.0,route 10.1.1.0 255.255.255.0,route 10.10.10.0 255.255.255.0,route 10.10.20.0 255.255.255.0,route 10.10.30.0 255.255.255.0,route 10.10.40.0 255.255.255.0,route 10.10.50.0 255.255.255.0,route-gateway 10.6.5.1,topology subnet,ping 10,ping-restart 120,ifconfig 10.6.5.2 255.255.255.0,peer-id 0,cipher AES-256-GCM' (status=1)
Sun Oct  6 14:48:21 2019 daemon.notice openvpn(server)[18386]: rw/<REMOVED>:56580 Data Channel: using negotiated cipher 'AES-256-GCM'
Sun Oct  6 14:48:21 2019 daemon.notice openvpn(server)[18386]: rw/<REMOVED>:56580 Outgoing Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
Sun Oct  6 14:48:21 2019 daemon.notice openvpn(server)[18386]: rw/<REMOVED>:56580 Incoming Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
root@OpenWrt.dmz:/etc/openvpn$ 

VPN client log:

Sun Oct  6 14:48:18 2019 OpenVPN 2.4.5 mips-openwrt-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [MH/PKTINFO] [AEAD]
Sun Oct  6 14:48:18 2019 library versions: OpenSSL 1.0.2t  10 Sep 2019, LZO 2.10
Sun Oct  6 14:48:18 2019 Outgoing Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
Sun Oct  6 14:48:18 2019 Outgoing Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication
Sun Oct  6 14:48:18 2019 Incoming Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
Sun Oct  6 14:48:18 2019 Incoming Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication
Sun Oct  6 14:48:18 2019 TCP/UDP: Preserving recently used remote address: [AF_INET]<REMOVED>:1194
Sun Oct  6 14:48:18 2019 Socket Buffers: R=[87380->87380] S=[16384->16384]
Sun Oct  6 14:48:18 2019 Attempting to establish TCP connection with [AF_INET]<REMOVED>:1194 [nonblock]
Sun Oct  6 14:48:19 2019 TCP connection established with [AF_INET]<REMOVED>:1194
Sun Oct  6 14:48:19 2019 TCP_CLIENT link local: (not bound)
Sun Oct  6 14:48:19 2019 TCP_CLIENT link remote: [AF_INET]<REMOVED>:1194
Sun Oct  6 14:48:19 2019 TLS: Initial packet from [AF_INET]<REMOVED>:1194, sid=1ce67d6e ca16beed
Sun Oct  6 14:48:20 2019 VERIFY OK: depth=1, CN=my.server.name
Sun Oct  6 14:48:20 2019 VERIFY KU OK
Sun Oct  6 14:48:20 2019 Validating certificate extended key usage
Sun Oct  6 14:48:20 2019 ++ Certificate has EKU (str) TLS Web Server Authentication, expects TLS Web Server Authentication
Sun Oct  6 14:48:20 2019 VERIFY EKU OK
Sun Oct  6 14:48:20 2019 VERIFY OK: depth=0, CN=server
Sun Oct  6 14:48:20 2019 Control Channel: TLSv1.2, cipher TLSv1/SSLv3 ECDHE-RSA-AES256-GCM-SHA384, 2048 bit RSA
Sun Oct  6 14:48:20 2019 [server] Peer Connection Initiated with [AF_INET]<REMOVED>:1194
Sun Oct  6 14:48:21 2019 SENT CONTROL [server]: 'PUSH_REQUEST' (status=1)
Sun Oct  6 14:48:21 2019 PUSH: Received control message: 'PUSH_REPLY,dhcp-option DNS 10.6.5.1,dhcp-option DOMAIN dmz,dhcp-option DOMAIN lan,dhcp-option DOMAIN home,dhcp-option DOMAIN servers.local,dhcp-option DOMAIN lit.int,dhcp-option DOMAIN rw,route 10.6.5.0 255.255.255.0,route 10.1.1.0 255.255.255.0,route 10.10.10.0 255.255.255.0,route 10.10.20.0 255.255.255.0,route 10.10.30.0 255.255.255.0,route 10.10.40.0 255.255.255.0,route 10.10.50.0 255.255.255.0,route-gateway 10.6.5.1,topology subnet,ping 10,ping-restart 120,ifconfig 10.6.5.2 255.255.255.0,peer-id 0,cipher AES-256-GCM'
Sun Oct  6 14:48:21 2019 OPTIONS IMPORT: timers and/or timeouts modified
Sun Oct  6 14:48:21 2019 OPTIONS IMPORT: --ifconfig/up options modified
Sun Oct  6 14:48:21 2019 OPTIONS IMPORT: route options modified
Sun Oct  6 14:48:21 2019 OPTIONS IMPORT: route-related options modified
Sun Oct  6 14:48:21 2019 OPTIONS IMPORT: --ip-win32 and/or --dhcp-option options modified
Sun Oct  6 14:48:21 2019 OPTIONS IMPORT: peer-id set
Sun Oct  6 14:48:21 2019 OPTIONS IMPORT: adjusting link_mtu to 1626
Sun Oct  6 14:48:21 2019 OPTIONS IMPORT: data channel crypto options modified
Sun Oct  6 14:48:21 2019 Data Channel: using negotiated cipher 'AES-256-GCM'
Sun Oct  6 14:48:21 2019 Outgoing Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
Sun Oct  6 14:48:21 2019 Incoming Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
Sun Oct  6 14:48:21 2019 TUN/TAP device tun0 opened
Sun Oct  6 14:48:21 2019 TUN/TAP TX queue length set to 100
Sun Oct  6 14:48:21 2019 do_ifconfig, tt->did_ifconfig_ipv6_setup=0
Sun Oct  6 14:48:21 2019 /sbin/ifconfig tun0 10.6.5.2 netmask 255.255.255.0 mtu 1500 broadcast 10.6.5.255
Sun Oct  6 14:48:21 2019 /sbin/route add -net 10.6.5.0 netmask 255.255.255.0 gw 10.6.5.1
Sun Oct  6 14:48:21 2019 /sbin/route add -net 10.1.1.0 netmask 255.255.255.0 gw 10.6.5.1
Sun Oct  6 14:48:21 2019 /sbin/route add -net 10.10.10.0 netmask 255.255.255.0 gw 10.6.5.1
Sun Oct  6 14:48:21 2019 /sbin/route add -net 10.10.20.0 netmask 255.255.255.0 gw 10.6.5.1
Sun Oct  6 14:48:21 2019 /sbin/route add -net 10.10.30.0 netmask 255.255.255.0 gw 10.6.5.1
Sun Oct  6 14:48:21 2019 /sbin/route add -net 10.10.40.0 netmask 255.255.255.0 gw 10.6.5.1
Sun Oct  6 14:48:21 2019 /sbin/route add -net 10.10.50.0 netmask 255.255.255.0 gw 10.6.5.1
Sun Oct  6 14:48:21 2019 Initialization Sequence Completed

I am perfectly able to ping the OpenVPNs LAN (and beyond - that is the other subnets):

root@OpenWrt:~# ping 10.1.1.1
PING 10.1.1.1 (10.1.1.1): 56 data bytes
64 bytes from 10.1.1.1: seq=0 ttl=64 time=45.021 ms
64 bytes from 10.1.1.1: seq=1 ttl=64 time=44.361 ms
64 bytes from 10.1.1.1: seq=2 ttl=64 time=42.608 ms
^C
--- 10.1.1.1 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 42.608/43.996/45.021 ms
root@OpenWrt:~# ping 10.10.20.1
PING 10.10.20.1 (10.10.20.1): 56 data bytes
64 bytes from 10.10.20.1: seq=0 ttl=63 time=58.309 ms
64 bytes from 10.10.20.1: seq=1 ttl=63 time=41.539 ms
64 bytes from 10.10.20.1: seq=2 ttl=63 time=39.577 ms
^C
--- 10.10.20.1 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 39.577/46.475/58.309 ms
root@OpenWrt:~# ping 10.10.40.1
PING 10.10.40.1 (10.10.40.1): 56 data bytes
64 bytes from 10.10.40.1: seq=0 ttl=63 time=45.812 ms
64 bytes from 10.10.40.1: seq=1 ttl=63 time=40.364 ms
64 bytes from 10.10.40.1: seq=2 ttl=63 time=38.943 ms
^C
--- 10.10.40.1 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 38.943/41.706/45.812 ms
root@OpenWrt:~# 

The other way round does not work however for the clients LAN (although the route is theoretically there) - pinging the client via its VPN IP works however perfectly fine:

root@OpenWrt.dmz:/etc/openvpn$ ping 10.10.60.1
PING 10.10.60.1 (10.10.60.1): 56 data bytes
^C
--- 10.10.60.1 ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss
root@OpenWrt.dmz:/etc/openvpn$ ping 10.6.5.2
PING 10.6.5.2 (10.6.5.2): 56 data bytes
64 bytes from 10.6.5.2: seq=0 ttl=64 time=44.596 ms
64 bytes from 10.6.5.2: seq=1 ttl=64 time=56.563 ms
64 bytes from 10.6.5.2: seq=2 ttl=64 time=38.460 ms
^C
--- 10.6.5.2 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 38.460/46.539/56.563 ms
root@OpenWrt.dmz:/etc/openvpn$ 

On the client i dont receive any packets (checked with tcpdump) when pinging 10.10.60.1, only when pinging 10.6.5.2 (the clients IP).

I simply don't understand the issue here, as the routes necessary seem to be present - first I thought I need to masquerade my outgoing packets from both sides (vpn and client), when it comes to accessing their LANs; This however, did not work as well.
traceroute 10.10.60.1 leads to nothing but stars (*) - I guess the VPN server does simply not know, where the paket should travel to (which route he needs to use)

I am a bit lost now and any help would be really much appreciated!

Thank you very much!

nobody? :frowning:

The default firewall configuration provides a flexible and scalable setup.
When you try to simplify it without deep enough understanding, it becomes incorrect.
A site-to-site connection assumes bidirectional communication that doesn't match your configuration.
And the way you handle the RELATED/ESTABLISHED is also incorrect.

In addition, these settings don't match:

It should be like this:
https://openwrt.org/docs/guide-user/services/vpn/openvpn/extra#site-to-site

1 Like

The default firewall configuration provides a flexible and scalable setup.
When you try to simplify it without deep enough understanding, it becomes incorrect.

That's the whole point of it - I want to understand it. Not by utilizing a "3rd-party tool" (uci), which does the magic for me then, instead of understanding the concept of routing and how things have to be/should be done on the "lower" level. If I would be after some easy to setup and run kind of thing, I would simply buy unifi or similar - but as I said, I want to learn.

I agree about the ifconfig-push command beeing wrong - I fixed it with

ifconfig-push 10.6.5.2 255.255.255.0
iroute 10.10.60.0 255.255.255.0

However, I couldn't find anything related to topology in the article you mentioned.
Unfortunately it didn't fix the issue I am having aswell.

Also: Could you tell me what's wrong with how I handle related and established connections?
Thanks!

Understanding firewall principles is a completely different and huge topic.
Using the OpenWrt firewall service is the simplest way to configure firewall properly.

The rule should apply to all interfaces and should be placed in the beginning of the chain, excluding prohibitive rules if any.

Why do you need those rules?
Remove them.

tcpdump -n -i any icmp

Understanding firewall principles is a completely different and huge topic.
Using the OpenWrt firewall service is the simplest way to configure firewall properly.
I guess we all started somewhere, don't we? :slight_smile:

The rule should apply to all interfaces and should be placed in the beginning of the chain, excluding prohibitive rules if any.

Okay, thanks. However, I would only see a performance benefit here, placing it at the very top (assuming, this might be the rule getting used/hit most). Or is there something I am missing? I only drop/reject packets at the very bottom, so this wouldn't cause any issues?

Why do you need those rules?
Remove them.

Actually I don't - as mentioned in my initial post: I thought I might need to masquerade my packets, but that didn't work out either (turns out, either works).

tcpdump -n -i any icmp

As I said in my initial post, only requests from the VPN server to the VPN client within the 10.6.5.0/24 range are getting through; Please see following output:
Output server:

root@OpenWrt.dmz:~$ ping 10.10.60.1
PING 10.10.60.1 (10.10.60.1): 56 data bytes
^C
--- 10.10.60.1 ping statistics ---
3 packets transmitted, 0 packets received, 100% packet loss
root@OpenWrt.dmz:~$ ping 10.6.5.2
PING 10.6.5.2 (10.6.5.2): 56 data bytes
64 bytes from 10.6.5.2: seq=0 ttl=64 time=40.526 ms
64 bytes from 10.6.5.2: seq=1 ttl=64 time=38.678 ms
64 bytes from 10.6.5.2: seq=2 ttl=64 time=42.415 ms
64 bytes from 10.6.5.2: seq=3 ttl=64 time=43.409 ms
64 bytes from 10.6.5.2: seq=4 ttl=64 time=42.341 ms
64 bytes from 10.6.5.2: seq=5 ttl=64 time=42.995 ms
64 bytes from 10.6.5.2: seq=6 ttl=64 time=42.421 ms
^C
--- 10.6.5.2 ping statistics ---
7 packets transmitted, 7 packets received, 0% packet loss
round-trip min/avg/max = 38.678/41.826/43.409 ms
root@OpenWrt.dmz:~$ 

Output client:

root@OpenWrt.rw:~$ tcpdump -n -i any icmp 
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
01:19:28.396741 IP 10.6.5.1 > 10.6.5.2: ICMP echo request, id 33120, seq 0, length 64
01:19:28.396901 IP 10.6.5.2 > 10.6.5.1: ICMP echo reply, id 33120, seq 0, length 64
01:19:29.395940 IP 10.6.5.1 > 10.6.5.2: ICMP echo request, id 33120, seq 1, length 64
01:19:29.396112 IP 10.6.5.2 > 10.6.5.1: ICMP echo reply, id 33120, seq 1, length 64
01:19:30.395320 IP 10.6.5.1 > 10.6.5.2: ICMP echo request, id 33120, seq 2, length 64
01:19:30.395493 IP 10.6.5.2 > 10.6.5.1: ICMP echo reply, id 33120, seq 2, length 64
01:19:31.395881 IP 10.6.5.1 > 10.6.5.2: ICMP echo request, id 33120, seq 3, length 64
01:19:31.396054 IP 10.6.5.2 > 10.6.5.1: ICMP echo reply, id 33120, seq 3, length 64
01:19:32.395648 IP 10.6.5.1 > 10.6.5.2: ICMP echo request, id 33120, seq 4, length 64
01:19:32.395821 IP 10.6.5.2 > 10.6.5.1: ICMP echo reply, id 33120, seq 4, length 64
01:19:33.396763 IP 10.6.5.1 > 10.6.5.2: ICMP echo request, id 33120, seq 5, length 64
01:19:33.396938 IP 10.6.5.2 > 10.6.5.1: ICMP echo reply, id 33120, seq 5, length 64
01:19:34.396022 IP 10.6.5.1 > 10.6.5.2: ICMP echo request, id 33120, seq 6, length 64
01:19:34.396213 IP 10.6.5.2 > 10.6.5.1: ICMP echo reply, id 33120, seq 6, length 64

Routes are still there ..
Server:

root@OpenWrt.dmz:~$ route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         <REMOVED>   0.0.0.0         UG    0      0        0 pppoe-wan
10.1.1.0        *               255.255.255.0   U     0      0        0 br-lan
10.6.5.0        *               255.255.255.0   U     0      0        0 tun0
10.10.10.0      openwrt-lan.dmz 255.255.255.0   UG    0      0        0 br-lan
10.10.20.0      openwrt-lan.dmz 255.255.255.0   UG    0      0        0 br-lan
10.10.30.0      openwrt-lan.dmz 255.255.255.0   UG    0      0        0 br-lan
10.10.40.0      10.1.1.4        255.255.255.0   UG    0      0        0 br-lan
10.10.60.0      10.6.5.2        255.255.255.0   UG    0      0        0 tun0
<REMOVED>   *               255.255.255.255 UH    0      0        0 pppoe-wan
root@OpenWrt.dmz:~$

Client:

root@OpenWrt.rw:~$ route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         192.168.178.1   0.0.0.0         UG    0      0        0 eth0.2
10.1.1.0        10.6.5.1        255.255.255.0   UG    0      0        0 tun0
10.6.5.0        10.6.5.1        255.255.255.0   UG    0      0        0 tun0
10.6.5.0        *               255.255.255.0   U     0      0        0 tun0
10.10.10.0      10.6.5.1        255.255.255.0   UG    0      0        0 tun0
10.10.20.0      10.6.5.1        255.255.255.0   UG    0      0        0 tun0
10.10.30.0      10.6.5.1        255.255.255.0   UG    0      0        0 tun0
10.10.40.0      10.6.5.1        255.255.255.0   UG    0      0        0 tun0
10.10.60.0      *               255.255.255.0   U     0      0        0 br-lan
192.168.178.0   *               255.255.255.0   U     0      0        0 eth0.2
root@OpenWrt.rw:~$ 

Thanks!

Check tcpdump on the VPN client, server and destination host to identify, which host drops the requests.
Test traceroute in both directions to make sure the VPN gateway matches the LAN gateway for both client and server side respectively, otherwise you need to add the necessary routes on your LAN gateway.

1 Like

I did that already. To make it not too complicated I only tried pinging the VPN client on the VPN clients LANs adress - same goes for traceroute. The paket never leaves the VPN server at all.

Ping from VPN server to VPN client on its LAN address (10.10.60.1) (tcpdump is from the VPN server - on the VPN client is nothing to see):

12:28:53.000398 IP 10.6.5.1 > 10.10.60.1: ICMP echo request, id 1636, seq 13, length 64
12:28:54.000495 IP 10.6.5.1 > 10.10.60.1: ICMP echo request, id 1636, seq 14, length 64
12:28:55.000658 IP 10.6.5.1 > 10.10.60.1: ICMP echo request, id 1636, seq 15, length 64
12:28:56.000821 IP 10.6.5.1 > 10.10.60.1: ICMP echo request, id 1636, seq 16, length 64
12:28:57.000987 IP 10.6.5.1 > 10.10.60.1: ICMP echo request, id 1636, seq 17, length 64
12:28:58.001149 IP 10.6.5.1 > 10.10.60.1: ICMP echo request, id 1636, seq 18, length 64
12:28:59.001314 IP 10.6.5.1 > 10.10.60.1: ICMP echo request, id 1636, seq 19, length 64

Traceroute from the VPN server to the VPN client on the VPN clients LAN (10.10.60.1):

root@OpenWrt.dmz:~$ traceroute 10.10.60.1
traceroute to 10.10.60.1 (10.10.60.1), 30 hops max, 38 byte packets
 1  *  *  *
 2

Then most likely it is dropped by the firewall for some reason.
It has been already suggested by @vgaetera to use the built-in firewall, I'll second that.
After you have an operational firewall you can see the rules with iptables-save and if you feel like experimenting you can do it with a working starting point.

1 Like

I did that beforehand exactly like that (in another topic I created).
Also:

root@OpenWrt.dmz:~$ iptables-save 
# Generated by iptables-save v1.6.2 on Wed Oct  9 15:51:07 2019
*nat
:PREROUTING ACCEPT [65845:7131262]
:INPUT ACCEPT [10898:1166522]
:OUTPUT ACCEPT [617:38879]
:POSTROUTING ACCEPT [119:5531]
-A POSTROUTING -o pppoe-wan -j MASQUERADE
COMMIT
# Completed on Wed Oct  9 15:51:07 2019
# Generated by iptables-save v1.6.2 on Wed Oct  9 15:51:07 2019
*mangle
:PREROUTING ACCEPT [72015372:71386055630]
:INPUT ACCEPT [437284:47533830]
:FORWARD ACCEPT [71572238:71320730101]
:OUTPUT ACCEPT [354551:43190418]
:POSTROUTING ACCEPT [71921103:71363112334]
-A FORWARD -o pppoe-wan -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu
COMMIT
# Completed on Wed Oct  9 15:51:07 2019
# Generated by iptables-save v1.6.2 on Wed Oct  9 15:51:07 2019
*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [234484:26257459]
:chn_reject - [0:0]
-A INPUT -p tcp -m tcp --dport 1194 -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -i br-lan -j ACCEPT
-A INPUT -s 10.1.1.2/32 -j ACCEPT
-A INPUT -p icmp -m icmp --icmp-type 8 -j ACCEPT
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p tcp -m tcp --dport 1905 -j ACCEPT
-A INPUT -i tun0 -j ACCEPT
-A INPUT -j LOG --log-prefix "iptables_INPUT_denied: " --log-level 7
-A INPUT -j chn_reject
-A FORWARD -p icmp -m icmp --icmp-type 8 -j ACCEPT
-A FORWARD -i tun0 -o br-lan -j ACCEPT
-A FORWARD -o tun0 -j ACCEPT
-A FORWARD -i br-lan -o pppoe-wan -j ACCEPT
-A FORWARD -i pppoe-wan -o br-lan -m conntrack --ctstate RELATED,ESTABLISHED,DNAT -j ACCEPT
-A FORWARD -j LOG --log-prefix "iptables_FORWARD_denied: " --log-level 7
-A FORWARD -j chn_reject
-A chn_reject -p tcp -j REJECT --reject-with tcp-reset
-A chn_reject -j REJECT --reject-with icmp-port-unreachable
COMMIT
# Completed on Wed Oct  9 15:51:07 2019
root@OpenWrt.dmz:~$ 

Specifically:

-A INPUT -j LOG --log-prefix "iptables_INPUT_denied: " --log-level 7
-A INPUT -j chn_reject

-A FORWARD -j LOG --log-prefix "iptables_FORWARD_denied: " --log-level 7
-A FORWARD -j chn_reject

I drop nothing before not logging it beforehand - I cannot see any packet beeing dropped regarding that.

MULTI: internal route 10.10.60.0/24 -> rw/RW_ADDR:RW_PORT
MULTI: Learn: 10.10.60.0/24 -> rw/RW_ADDR:RW_PORT
2 Likes

Okay, I really don't know how I could miss that one - THANKS!
Now everything works as expected:

root@OpenWrt.dmz:~$ ping 10.10.60.1
PING 10.10.60.1 (10.10.60.1): 56 data bytes
64 bytes from 10.10.60.1: seq=0 ttl=64 time=87.832 ms
64 bytes from 10.10.60.1: seq=1 ttl=64 time=65.328 ms
64 bytes from 10.10.60.1: seq=2 ttl=64 time=39.222 ms
1 Like

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.