IPsec traffic not traversing OpenWrt

Hello everyone,

I am scratching my head for a few weeks on how to set up a Routed site-to-site VPN between pfSense and openwrt. :tired_face:

The connection is establishing and according to my troubleshooting it seems that traffic traverses the tunnel but it is dropped on the openwrt side.

I will provide the configuration at the openwrt side of the tunnel :point_down:

:arrow_right: OpenWrt

/etc/swanctl/swanctl.conf

connections {
        bypass {
                remote_addrs = 127.0.0.1
                children {
                        bypasslan {
                                local_ts = 10.1.1.0/24
                                remote_ts = 10.1.1.0/24
                                mode = pass
                                start_action = trap
                        }
                }
        }
        router01 {
                fragmentation = yes
                unique = replace
                version = 2
                proposals = aes256-sha256-modp2048
                dpd_delay = 10s
                dpd_timeout = 60s
                rekey_time = 25920s
                reauth_time = 0s
                over_time = 2880s
                rand_time = 2880s
                encap = no
                mobike = no
                local_addrs = router01.viewdns.net
                remote_addrs = 94.228.37.60
                local {
                        id = fqdn:ipsec-home
                        auth = psk
                }
                remote {
                        id = fqdn:ipsec-tucana
                        auth = psk
                }
                children {
                        tucana {
                                close_action = start
                                dpd_action = restart
                                #mode = tunnel
                                policies = no
                                life_time = 3600s
                                rekey_time = 3240s
                                rand_time = 360s
                                start_action = trap
                                remote_ts = 192.168.201.1/30
                                local_ts = 192.168.201.2/30
                                #reqid = 100
                                mark_in = 32
                                mark_out = 42
                                esp_proposals = aes256gcm128-modp2048
                        }
                }
        }
}
secrets {
        ike-0 {
                secret = passwd
                id-0 = %any
                id-1 = fqdn:ipsec-tucana
        }
}

/etc/strongswan.conf

# CUSTOM from pfSense
starter {
        load_warning = no
}

charon {
        # number of worker threads in charon
        threads = 16
        ikesa_table_size = 32
        ikesa_table_segments = 4
        init_limit_half_open = 1000
        install_routes = no
        install_virtual_ip = no # not in pfSense side. Added from internet tutorials.
        load_modular = yes
        ignore_acquire_ts = yes
        cisco_unity = no
        syslog {
                identifier = charon
                # log everything under daemon since it ends up in the same place regardless with our syslog.conf
                daemon {
                        ike_name = yes
                        app = -1
                        asn = -1
                        cfg = -1
                        chd = -1
                        dmn = -1
                        enc = -1
                        esp = -1
                        ike = -1 # set to 2 to troubleshoot
                        imc = -1
                        imv = -1
                        job = -1
                        knl = -1 # set to 2 to troubleshoot
                        lib = -1
                        mgr = -1
                        net = 2
                        pts = -1
                        tls = -1
                        tnc = -1
                }
                # disable logging under auth so logs aren't duplicated
                auth {
                        default = -1 # set to 2 for troubleshooting; -1 to supress
                        ike = -1
                }
        }
        plugins {
                #kernel-netlink{
                #       fwmark = !0x42
                #}
                include strongswan.d/charon/*.conf
        }
}
include strongswan.d/*.conf
include /var/ipsec/strongswan.conf

/etc/config/network

...
config interface 'ipsec_vlan201'
        option ifname 'ip_vti1'
        option proto 'static'
        option ipaddr '192.168.201.2'
        option netmask '255.255.255.252'
...

/etc/config/firewall

...
config rule
        option name 'Allow-IPSec-IKE-input'
        option target 'ACCEPT'
        list proto 'udp'
        option src 'wan'
        option dest_port '500 4500'

config rule
        option name 'Allow-IPSec-ESP-input'
        option target 'ACCEPT'
        list proto 'esp'
        option src 'wan'

config rule
        option name 'Allow-IPSec-Auth-Header-input'
        option target 'ACCEPT'
        list proto 'ah'
        option src 'wan'
...

config zone
        option name 'wan'
        option input 'REJECT'
        option output 'ACCEPT'
        option forward 'REJECT'
        option masq '1'
        option mtu_fix '1'
        option network 'wan'
        option extra_src '-m policy --dir in --pol none'
        option extra_dest '-m policy --dir out --pol none'

config zone
        option name 'vlan5'
        option network 'vlan5 eth0_VLAN5'
        option output 'ACCEPT'
        option input 'ACCEPT'
        option family 'ipv4'
        option forward 'ACCEPT'
        option extra_src '-m policy --dir in --pol none'
        option extra_dest '-m policy --dir out --pol none'
...

config zone
        option family 'ipv4'
        option output 'ACCEPT'
        option input 'ACCEPT'
        option forward 'ACCEPT'
        option log '1'
        option name 'vlan201'
        option log_limit '2/minute'
        option network 'ipsec_vlan201'
        option extra_src '-m policy --dir in --pol ipsec --proto esp'
        option extra_dest '-m policy --dir out --pol ipsec --proto esp'
        list device 'ip_vti1'

I have added the extra iptables rules to non vpn zones as suggested on this post

I am able to connect with the configuration above as seen below :point_down:

root@R1 /etc/config > swanctl --load-all
root@R1 /etc/config > swanctl --initiate -c tucana

:green_circle: ipsec statusall

Status of IKE charon daemon (strongSwan 5.8.2, Linux 4.14.221, armv7l):
  uptime: 2 hours, since Aug 08 22:05:13 2021
  worker threads: 10 of 16 idle, 6/0/0/0 working, job queue: 0/0/0/0, scheduled: 5
  loaded plugins: charon test-vectors ldap pkcs11 aes des blowfish rc2 sha2 sha1 md4 md5 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pgp dnskey sshkey pem openssl gcrypt af-alg fips-prf gmp curve25519 agent xcbc cmac hmac ctr ccm gcm curl mysql sqlite attr kernel-netlink resolve socket-default connmark forecast farp stroke vici smp updown eap-identity eap-md5 eap-mschapv2 eap-radius eap-tls xauth-generic xauth-eap dhcp whitelist led duplicheck addrblock unity
Listening IP addresses:
  192.168.201.2
  192.168.30.1
  192.168.40.1
  192.168.50.1
  192.168.10.1
  192.168.20.1
  192.168.5.1
  192.168.6.1
  192.168.60.1
  194.75.11.234
Connections:
      bypass:  %any...127.0.0.1  IKEv1/2
      bypass:   local:  uses any authentication
      bypass:   remote: uses any authentication
   bypasslan:   child:  10.1.1.0/24 === 10.1.1.0/24 PASS
    router01:  router01.viewdns.net...94.228.37.60  IKEv2, dpddelay=10s
    router01:   local:  [ipsec-home] uses pre-shared key authentication
    router01:   remote: [ipsec-tucana] uses pre-shared key authentication
      tucana:   child:  192.168.201.0/30 === 192.168.201.0/30 TUNNEL, dpdaction=restart
Shunted Connections:
   bypasslan:  10.1.1.0/24 === 10.1.1.0/24 PASS
Routed Connections:
      tucana{1}:  CREATED, TUNNEL, reqid 1
      tucana{1}:   192.168.201.0/30 === 192.168.201.0/30
Security Associations (1 up, 0 connecting):
    router01[3]: ESTABLISHED 51 minutes ago, 194.75.11.234[ipsec-home]...94.228.37.60[ipsec-tucana]
    router01[3]: IKEv2 SPIs: 4a7b5f08f680f34e_i* ee6f0c589a57b258_r, rekeying in 5 hours
    router01[3]: IKE proposal: AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048
      tucana{5}:  INSTALLED, TUNNEL, reqid 1, ESP SPIs: cef99bfc_i c9d7dc00_o
      tucana{5}:  AES_GCM_16_256/MODP_2048, 9599 bytes_i, 0 bytes_o, rekeying in 47 minutes
      tucana{5}:   192.168.201.0/30 === 192.168.201.0/30

The interface ip_vti1 was addes with the command below :point_down:

root@R1 /etc/config > ip tunnel add ip_vti1 local 194.75.11.234 remote 94.228.37.60 mode vti ikey 32 okey 42
root@R1 /etc/config > sysctl -w net.ipv4.conf.ip_vti1.disable_policy=1
root@R1 /etc/config > ip link set ip_vti1 up

root@R1 /etc/config > ip -c a
30: ip_vti1@NONE: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1472 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ipip 194.75.11.234 peer 94.228.37.60
    inet 192.168.201.2/30 brd 192.168.201.3 scope global ip_vti1
       valid_lft forever preferred_lft forever
    inet6 fe80::200:5efe:c24b:bea/64 scope link 
       valid_lft forever preferred_lft forever

A route was added as :point_down:

root@R1 /etc/config > ip route
...
192.168.201.0/30 dev ip_vti1 proto kernel scope link src 192.168.201.2 

I have captured ESP packets on my WAN :point_down:

root@R1 /etc/config > tcpdump -n -v -i pppoe-wan proto \\esp
tcpdump: listening on pppoe-wan, link-type LINUX_SLL (Linux cooked v1), capture size 262144 bytes
00:58:19.921492 IP (tos 0x0, ttl 54, id 46196, offset 0, flags [none], proto ESP (50), length 84)
    94.228.37.60 > 194.75.11.234: ESP(spi=0xcef99bfc,seq=0x5ea), length 64
00:58:20.463300 IP (tos 0x0, ttl 54, id 29878, offset 0, flags [none], proto ESP (50), length 84)
    94.228.37.60 > 194.75.11.234: ESP(spi=0xcef99bfc,seq=0x5eb), length 64
00:58:21.004909 IP (tos 0x0, ttl 54, id 465, offset 0, flags [none], proto ESP (50), length 84)
    94.228.37.60 > 194.75.11.234: ESP(spi=0xcef99bfc,seq=0x5ec), length 64
00:58:21.536088 IP (tos 0x0, ttl 54, id 8152, offset 0, flags [none], proto ESP (50), length 84)
    94.228.37.60 > 194.75.11.234: ESP(spi=0xcef99bfc,seq=0x5ed), length 64
00:58:22.077409 IP (tos 0x0, ttl 54, id 29796, offset 0, flags [none], proto ESP (50), length 84)
    94.228.37.60 > 194.75.11.234: ESP(spi=0xcef99bfc,seq=0x5ee), length 64
00:58:22.618782 IP (tos 0x0, ttl 54, id 29587, offset 0, flags [none], proto ESP (50), length 84)
    94.228.37.60 > 194.75.11.234: ESP(spi=0xcef99bfc,seq=0x5ef), length 64
00:58:23.137175 IP (tos 0x0, ttl 54, id 21944, offset 0, flags [none], proto ESP (50), length 84)
    94.228.37.60 > 194.75.11.234: ESP(spi=0xcef99bfc,seq=0x5f0), length 64
00:58:23.678087 IP (tos 0x0, ttl 54, id 31737, offset 0, flags [none], proto ESP (50), length 84)
    94.228.37.60 > 194.75.11.234: ESP(spi=0xcef99bfc,seq=0x5f1), length 64
00:58:24.186914 IP (tos 0x0, ttl 54, id 33641, offset 0, flags [none], proto ESP (50), length 84)
    94.228.37.60 > 194.75.11.234: ESP(spi=0xcef99bfc,seq=0x5f2), length 64

Also a capture from a ping to the IP on the other side of the tunnel on the iface ip_vti1 :point_down:

root@R1 /root > ping -I ip_vti1 192.168.201.1
PING 192.168.201.1 (192.168.201.1): 56 data bytes
no replies
...




root@R1 /etc/config > tcpdump -n -v -i ip_vti1
tcpdump: listening on ip_vti1, link-type RAW (Raw IP), capture size 262144 bytes
01:00:17.221710 IP (tos 0x0, ttl 64, id 2874, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.201.2 > 192.168.201.1: ICMP echo request, id 18761, seq 0, length 64
01:00:18.221836 IP (tos 0x0, ttl 64, id 2962, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.201.2 > 192.168.201.1: ICMP echo request, id 18761, seq 1, length 64
01:00:19.221920 IP (tos 0x0, ttl 64, id 3050, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.201.2 > 192.168.201.1: ICMP echo request, id 18761, seq 2, length 64
01:00:20.221997 IP (tos 0x0, ttl 64, id 3083, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.201.2 > 192.168.201.1: ICMP echo request, id 18761, seq 3, length 64
01:00:21.222136 IP (tos 0x0, ttl 64, id 3145, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.201.2 > 192.168.201.1: ICMP echo request, id 18761, seq 4, length 64
01:00:22.222282 IP (tos 0x0, ttl 64, id 3217, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.201.2 > 192.168.201.1: ICMP echo request, id 18761, seq 5, length 64
01:00:23.222431 IP (tos 0x0, ttl 64, id 3280, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.201.2 > 192.168.201.1: ICMP echo request, id 18761, seq 6, length 64
^C
7 packets captured
7 packets received by filter
0 packets dropped by kernel
17 packets dropped by interface


I have noticed a weird behaviour on the interface. The command ip -s tunnel show some statistics of the tunnel and ip_vti1 has errors on RX that looks like the gateway monitoring pings coming from the pfSense and on TX my pings that's errors out with NoRoute. :point_down:

root@R1 /etc/config > ip -s tunnel
ip_vti1: ip/ip remote 94.228.37.60 local 194.75.11.234 ttl inherit ikey 32 okey 42
RX: Packets    Bytes        Errors CsumErrs OutOfSeq Mcasts
    0          0            14640  0        0        0       
TX: Packets    Bytes        Errors DeadLoop NoRoute  NoBufs
    0          0            1550   0        1550     0

:red_circle: I believe I have provided a good picture of my problem and I would appreciate if anyone with more experience could have a look into my setup and suggest about what to look next cause I can not see where the problem is.

Many thanks :+1:

After further investigation with the help of a friend some new interesting findings.

The package ip-full was not installed therefore it's been installed and issuing the command ip xfrm policy the route for ipsec was not installed. I will reboot the router and try to re-establish the tunnel after installing the ip-full package.

root@R1 /root > ip xfrm policy
src 10.1.1.0/24 dst 10.1.1.0/24 
        dir fwd priority 175423 
src 10.1.1.0/24 dst 10.1.1.0/24 
        dir in priority 175423 
src 10.1.1.0/24 dst 10.1.1.0/24 
        dir out priority 175423 
src 0.0.0.0/0 dst 0.0.0.0/0 
        socket in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
        socket out priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
        socket in priority 0 
src 0.0.0.0/0 dst 0.0.0.0/0 
        socket out priority 0 
src ::/0 dst ::/0 
        socket in priority 0 
src ::/0 dst ::/0 
        socket out priority 0 
src ::/0 dst ::/0 
        socket in priority 0 
src ::/0 dst ::/0 
        socket out priority 0

After a reboot the logs are showing :point_down:

:arrow_right: OpenWrt

Mon Aug  9 18:07:18 2021 daemon.info ipsec: 11[KNL] <router01|1> unable to query policy 192.168.201.0/30 === 192.168.201.0/30 fwd (mark 32/0xffffffff)
Mon Aug  9 18:07:18 2021 daemon.info ipsec: 11[KNL] <router01|1> querying SAD entry with SPI c5be4e3d
Mon Aug  9 18:07:28 2021 daemon.info ipsec: 08[KNL] <router01|1> querying policy 192.168.201.0/30 === 192.168.201.0/30 in (mark 32/0xffffffff)
Mon Aug  9 18:07:28 2021 daemon.info ipsec: 08[KNL] <router01|1> querying policy failed: No such file or directory (2)
Mon Aug  9 18:07:28 2021 daemon.info ipsec: 08[KNL] <router01|1> unable to query policy 192.168.201.0/30 === 192.168.201.0/30 in (mark 32/0xffffffff)
Mon Aug  9 18:07:28 2021 daemon.info ipsec: 08[KNL] <router01|1> querying policy 192.168.201.0/30 === 192.168.201.0/30 fwd (mark 32/0xffffffff)
Mon Aug  9 18:07:28 2021 daemon.info ipsec: 08[KNL] <router01|1> querying policy failed: No such file or directory (2)
Mon Aug  9 18:07:28 2021 daemon.info ipsec: 08[KNL] <router01|1> unable to query policy 192.168.201.0/30 === 192.168.201.0/30 fwd (mark 32/0xffffffff)
Mon Aug  9 18:07:28 2021 daemon.info ipsec: 08[KNL] <router01|1> querying SAD entry with SPI c5be4e3d

:arrow_right: pfSense

Aug 9 17:07:13 	charon 	14960 	10[KNL] <con1000|74> querying SAD entry with SPI ca2a7433
Aug 9 17:07:13 	charon 	14960 	10[KNL] <con1000|74> querying policy 192.168.201.0/30|/0 === 192.168.201.0/30|/0 in failed, not found
Aug 9 17:07:13 	charon 	14960 	10[KNL] <con1000|74> querying policy 192.168.201.0/30|/0 === 192.168.201.0/30|/0 in
Aug 9 17:07:03 	charon 	14960 	10[KNL] <con1000|74> querying SAD entry with SPI ca2a7433
Aug 9 17:07:03 	charon 	14960 	10[KNL] <con1000|74> querying policy 192.168.201.0/30|/0 === 192.168.201.0/30|/0 in failed, not found
Aug 9 17:07:03 	charon 	14960 	10[KNL] <con1000|74> querying policy 192.168.201.0/30|/0 === 192.168.201.0/30|/0 in
Aug 9 17:07:03 	charon 	14960 	08[KNL] <con1000|74> querying policy 192.168.201.0/30|/0 === 192.168.201.0/30|/0 out failed, not found
Aug 9 17:07:03 	charon 	14960 	08[KNL] <con1000|74> querying policy 192.168.201.0/30|/0 === 192.168.201.0/30|/0 out
Aug 9 17:07:03 	charon 	14960 	08[KNL] <con1000|74> querying SAD entry with SPI c6acb857
Aug 9 17:07:03 	charon 	14960 	08[KNL] <con1000|74> querying SAD entry with SPI ca2a7433
Aug 9 17:06:58 	charon 	14960 	07[KNL] <con1000|74> querying policy 192.168.201.0/30|/0 === 192.168.201.0/30|/0 out failed, not found
Aug 9 17:06:58 	charon 	14960 	07[KNL] <con1000|74> querying policy 192.168.201.0/30|/0 === 192.168.201.0/30|/0 out
Aug 9 17:06:58 	charon 	14960 	07[KNL] <con1000|74> querying SAD entry with SPI c6acb857
Aug 9 17:06:58 	charon 	14960 	07[KNL] <con1000|74> querying SAD entry with SPI ca2a7433 

Either router is complaining about routes not installed. It seems that there is an issue with the marking in the kernel. Strongswan's documentation suggests a 42 mark however the tutorial that I have followed is using mark_in = 32 mark_out = 42.

I will try to change the kernel mark.

Hi @VincentR,

I saw in some threads that you have an ipsec tunnel up. Any insights about what to look in my setup :point_up: ?

Many thanks.

Hi @h0ppus, as much as I do have an IPSec VTI tunnel running, I don't remember all the magic that was involved in getting it there.

I can't really compare your commands and mine as I use Connection specific VTI Devices so things like mark, IP address, routes are taken from that updown script.

I do have this in my up/down script about martian filtering which doesn't seem to be there for you (unsure if that comes into play as it's also not in the documentation except for the GRE section):
sysctl -w net.ipv4.conf.ip_vti1.rp_filter=0

Other than that, maybe check your routing tables.

ip rule # to see where things go
ip route show table 220 # where some things typically go from the previous command

I'd recommend you tcpdump to a file which you scp and open in wireshark rather than just looking at the command line. You may be able to see a bunch more packets getting lost (and don't limit yourself to one interface).

Hi @VincentR

Thanks for replying. :grinning_face_with_smiling_eyes:

I have created the updown script too. But weirdly it creates the interface and does add the routes and local/remote IP addresses. :man_facepalming:t5:


I can confirm that rp_filter is set 0 for the ipsec interface. :+1:t5:


I have tried to add routes to table 220, but it breaks my network. The internet stops working, the router kind of freezes and it has to be rebooted. :frowning_face:


That's one thing that I did not do. :neutral_face:


Anyway, I have given up setting up the tunnel between home and my private cloud and I created a LAB to play around with different settings. It will be much easier to run captures and test all sorts of configurations without breaking my network and have wife/flatmate complaining about no connectivity.

The pfSense and OpenWrt are set and I am starting to create the ipsec configuration. I hopefully going to be able to figure out what's wrong and will update this thread/write a blog post to document the setup.

Wish me luck :crossed_fingers:

Best of luck! I suspect you're so close that it will be something simple like a missing firewall rule. If you do get a blog out post about your setup, you're going to help a lot of people out there. A secondary blog post could be how you did your troubleshooting as it seems all of us get stuck at some stage with missing packets (I definitely did and tried multiple things until it worked and then happily left it alone - possibly without understanding what made the difference)

Quick update.

I have been able to replicate the same problem that was happening with my live environment :

pfSense pings traverses the tunnel but hit the Openwrt VTI interface in error and pings from Openwrt the same :point_down:t5:

root@OpenWrt:~# ip -s tunnel
ip_vti0: ip/ip remote any local any ttl inherit nopmtudisc key 0
RX: Packets    Bytes        Errors CsumErrs OutOfSeq Mcasts
    0          0            0      0        0        0       
TX: Packets    Bytes        Errors DeadLoop NoRoute  NoBufs
    0          0            0      0        0        0     
ip_vti1: ip/ip remote 192.168.45.10 local 192.168.45.20 ttl inherit key 42
RX: Packets    Bytes        Errors CsumErrs OutOfSeq Mcasts
    0          0            46346  0        0        0       
TX: Packets    Bytes        Errors DeadLoop NoRoute  NoBufs
    0          0            48406  0        48406    0

I have set every strongswan subsystem log to 4 and analysed it, but nothing obvious.

By the way, I have captured the ESP packets with wireshark but could not find the authentication key to decrypt the packet and inspect how it is arriving from pfSense. Strongswan has a module save-key that dumps the keys to a file, but the OpenWRT version was not compiled with such plugin. :man_shrugging:t5: ( A potential feature request for future releases ).

I am starting to be suspicious of a bug with strongswan or with OpenWRT and I will test the latest ( 21.02.0-rc4 ) version to see if it will work.

And, I will also spin another pfSense VM to test IPSec routed site-to-site between them to make sure that the pfSense is working as expected.

I am pulling my hair out in frustration at this point :joy:

Another quick update for those following this thread.

I have been able to decrypt the ESP packets coming from my pfSense and here's a quick guide on how to decrypt ESP packets.

First, we need the decryption keys that can be obtained with the command :

root@OpenWrt:~# ip xfrm state
src 192.168.45.20 dst 192.168.45.10
        proto esp spi 0xc8f1adb8 reqid 1 mode tunnel
        replay-window 0 flag af-unspec
        mark 0x2a/0xffffffff 
        aead rfc4106(gcm(aes)) 0xc48558bac5431efbd1b2eecb051c717be79c78d7ccad101cbcae4c463eb80d4acb9XXXXX 128
        anti-replay context: seq 0x0, oseq 0x0, bitmap 0x00000000
src 192.168.45.10 dst 192.168.45.20
        proto esp spi 0xc42d8c46 reqid 1 mode tunnel
        replay-window 32 flag af-unspec
        aead rfc4106(gcm(aes)) 0x19235112af76c17f7965f42263ddaf8040ee7dbcde7382f2f65a4eadbf06b0e7048XXXXX 128
        anti-replay context: seq 0x454, oseq 0x0, bitmap 0xffffffff

rfc4106(gcm(aes)) - is the encryption algorithm used.
0x1923.... - is the decryption key.
proto esp spi - The Security Parameter Index (SPI) is an identification tag added to the header while using IPsec for tunnelling

In wireshark go to Edit > Preferences > Protocols > ESP and mark the options as :

Click on Edit... and fill as :

I am decrypting in only one direction but if you can fill the details from the other direction if you wish.
Hit OK and the packets should be decrypted as seen below :

By the way, this dump helped a lot. I now know that the packets are arriving in the VTI device with the correct IP addresses and being dropped by the interface, since the dump above is from the WAN and with ip -s tunnel I can see errors on the VTI interface.

root@OpenWrt:~# ip -s tunnel
ip_vti1: ip/ip remote 192.168.45.10 local 192.168.45.20 ttl inherit key 42
RX: Packets    Bytes        Errors CsumErrs OutOfSeq Mcasts
    0          0            3175   0        0        0       
TX: Packets    Bytes        Errors DeadLoop NoRoute  NoBufs
    0          0            3239   0        3239     0

:point_right:t5: The troubleshooting continues ...

2 Likes

nice stuff! my money is still on firewall :face_with_monocle:

just passing by here to make sure you looked into setting the correct mtu on your VTI interface (this could see packets dropped).

Yeah, I have adjusted the MTU. It is currently set to 1436 :point_down:t5:

7: ip_vti1@NONE: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1436 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/ipip 192.168.45.30 peer 192.168.45.10 promiscuity 0 minmtu 0 maxmtu 0 
    vti remote 192.168.45.10 local 192.168.45.30 ikey 0.0.0.42 okey 0.0.0.42 addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65

By the way, I am currently using an Arch linux instead of openwrt, because I could not get NFLOG to work in openWRT.

I have managed to get pings from pfSense to reach the other side of the tunnel without being in error changing the remote_ts to 0.0.0.0/0 , but the packets from the Linux are failing with Errors NoRoute.

It seems that it is related to a routing loop, however I could not figure out how to fix it.

ip_vti1: ip/ip remote 192.168.45.10 local 192.168.45.30 ttl inherit nopmtudisc key 42
RX: Packets    Bytes        Errors CsumErrs OutOfSeq Mcasts
    58586      4921224      0      0        0        0       
TX: Packets    Bytes        Errors DeadLoop NoRoute  NoBufs
    0          0            121816 0        121816   0  


Mangle OUTPUT chain

The packets encapsulated by IPSec have the wrong destination and it seems that are not send through the tunnel. :point_up:t5:

I have asked for some help in the strongswan email list :point_down:t5:
https://lists.strongswan.org/pipermail/users/2021-August/015080.html

I hope someone there is able to help.

@h0ppus link to a recently posted blog post about a working IPSec setup. Probably worth a read.

1 Like

Thanks !!

I am going to read it.

I have still not been able to have traffic flowing through the tunnel.

:point_up:t5: an illustration of what I want to achieve. Of course in the place of the arch linux the end goal is to have an openWRT box, but for testing it makes no difference since they all use the stronSwan library.