TLS Failure on particular sites

I have a very strange problem, which may well not actually be a Lede problem, but I would be grateful for any help in diagnosing it.
When I go to https://vehicletax.service.gov.uk/ in Firefox it shows 'Performing a TLS Handshake to vehicletax.service.gov.uk' for a little while then fails saying 'Secure Connection Failed, The connection to the server was reset while the page was loading'. A similar thing happens in Google Chrome. Also it happens on two machines, one running Ubuntu 18.04 and the other 18.10.

However, if I connect via a VPN that I run (in the UK) then it connects ok, which suggests it is something to do with the router or connection to the internet. Also if I connect my phone to the router wifi then it also has the problem, but if I connect the phone using mobile data then it is ok.

I am running LEDE Reboot 17.01.2 r3435-65eec8bd5f on a Plusnet router which is actually a BT Home Hub 5A. I have not touched the router for months and this problem has only just appeared. I have tried a reboot.

I have had a go with tcpdump on the PC and using sudo tcpdump host vehicletax.service.gov.uk I see this. (I have inserted line feeds between each transaction to make it a bit easier to read).

11:44:04.711668 IP tigger.33584 > 107.162.132.57.https: Flags [S], seq 1219288911, win 29200, options [mss 1460,sackOK,TS val 270762524 ecr 0,nop,wscale 7], length 0

11:44:04.725582 IP 107.162.132.57.https > tigger.33584: Flags [S.], seq 2537281460, ack 1219288912, win 4356, options [mss 1460,sackOK,eol], length 0

11:44:04.725596 IP tigger.33584 > 107.162.132.57.https: Flags [.], ack 1, win 29200, length 0

11:44:04.725764 IP tigger.33584 > 107.162.132.57.https: Flags [P.], seq 1:235, ack 1, win 29200, length 234

11:44:04.739037 IP 107.162.132.57.https > tigger.33584: Flags [.], ack 235, win 4590, length 0

At this point there is several seconds wait

11:44:08.738918 IP 107.162.132.57.https > tigger.33584: Flags [R.], seq 1, ack 235, win 4590, length 0

11:44:08.739859 IP tigger.33590 > 107.162.132.57.https: Flags [S], seq 3421144022, win 29200, options [mss 1460,sackOK,TS val 270766552 ecr 0,nop,wscale 7], length 0

11:44:08.751098 IP 107.162.132.57.https > tigger.33590: Flags [S.], seq 2514563206, ack 3421144023, win 4356, options [mss 1460,sackOK,eol], length 0

11:44:08.751163 IP tigger.33590 > 107.162.132.57.https: Flags [.], ack 1, win 29200, length 0

11:44:08.751774 IP tigger.33590 > 107.162.132.57.https: Flags [P.], seq 1:235, ack 1, win 29200, length 234

11:44:08.762568 IP 107.162.132.57.https > tigger.33590: Flags [.], ack 235, win 4590, length 0

Then there is another wait and the last five lines are repeated. It repeats this a few more times then fails.

When I connect through the VPN and repeat the exercise I see

11:35:24.829072 IP tigger.55710 > 107.162.132.57.https: Flags [S], seq 822655265, win 29200, options [mss 1460,sackOK,TS val 801822720 ecr 0,nop,wscale 7], length 0

11:35:24.862589 IP 107.162.132.57.https > tigger.55710: Flags [S.], seq 2005228617, ack 822655266, win 4104, options [mss 1368,sackOK,eol], length 0

11:35:24.862608 IP tigger.55710 > 107.162.132.57.https: Flags [.], ack 1, win 29200, length 0

11:35:24.862784 IP tigger.55710 > 107.162.132.57.https: Flags [P.], seq 1:235, ack 1, win 29200, length 234

11:35:24.888874 IP 107.162.132.57.https > tigger.55710: Flags [.], ack 235, win 4338, length 0

This time there is no delay at this point

11:35:24.890242 IP 107.162.132.57.https > tigger.55710: Flags [P.], seq 1:103, ack 235, win 4338, length 102

11:35:24.890250 IP tigger.55710 > 107.162.132.57.https: Flags [.], ack 103, win 29200, length 0

and the communication continues.
I have run tcpdump in the router watching the internet port and it appears to show the same packets.
With my limited knowledge it looks as if there is a packet missing from the server back to me, but hopefully someone with more knowledge will be able to throw some more light on this.

Tested from 2 OpenWrt devices. The site loads from the US.

Perhaps it's a DNS lookup or scripts...are you running Adblockers or things of any kind?

As you're comfortable with tcpdump, you might try wireshark, as it provides good information about what is going on with the handshake, including being able to inspect the certificate provided.

1 Like

I have been running pihole (in a pi, not the router) but I have disabled it (and checked it is disabled). It can be seen from the tcpdump that it is resolving the name ok.

That's a good idea, though I wouldn't exactly say I was comfortable with tcpdump, having looked at a couple of tutorials I managed to get it do something that seemed useful.
Having done something similar with wireshark it seems that what is happening is the PC is sending SYN, the server replies with SYN, ACK and the PC replies with ACK. Then the PC sends a TLSv1 Client Hello and the server replies ACK, but then nothing else happens till the server sends RST (reset?) and it starts again.
When I connect through the vpn then after the server ACK the server sends Server Hello and it all gets going. So for some reason either the server is not sending the Server Hello or it is getting lost along the way somehow. I will have another go at looking at what goes out the WAN port, though when I looked before it did not appear that the Server Hello was there.

@openwrt-router:

wget --no-check-certificate -O /dev/null https://vehicletax.service.gov.uk/
opkg update; opkg install ca-bundle; wget -O /dev/null https://vehicletax.service.gov.uk/

Can you explain the purpose of that please? I don't like to do things without having at least a vague idea of what the purpose is. I assumed that when I try to access a secure site from my PC that the router just routes the messages out and in again. I assumed that it did not take part in checking the certificate.
When I run

wget --no-check-certificate https://vehicletax.service.gov.uk/

in the PC I see

--2018-10-24 09:26:14--  https://vehicletax.service.gov.uk/
Resolving vehicletax.service.gov.uk (vehicletax.service.gov.uk)... 107.162.132.57
Connecting to vehicletax.service.gov.uk (vehicletax.service.gov.uk)|107.162.132.57|:443... connected.
Unable to establish SSL connection.

When I run it in the router I see

wget: SSL support not available, please install one of the libustream-ssl-* libraries as well as the ca-bundle and ca-certificates packages.

This is the only site that I have any such problem with (on the PC).

The purpose is to localize the issue.

It's not only routing, but also applying iptables rules to forwarded traffic.

OK, understood. Has the result thrown any light on the issue?

ensure mss-clamping is enabled on wan and then try to reduce wan-interface mtu (by 10, 20 ...)

1 Like

This means your computer cannot access the HTTPS site properly, @fuller offered some suggestions.

Thanks for the suggestions. mss clamping is enabled on the wan interface.
I tried mtu 1490, 1480, 1450 and 1400 but it made no difference, still the same problem.
With mtu on default traceroute --mtu gives

$ traceroute vehicletax.service.gov.uk --mtu
traceroute to vehicletax.service.gov.uk (107.162.132.57), 30 hops max, 65000 byte packets
 1  router.local (192.168.49.1)  0.566 ms F=1500  0.615 ms  0.537 ms
 2  * F=1492 * *
 3  * * *
 4  172.17.6.205 (172.17.6.205)  5.311 ms  5.227 ms  5.145 ms
 5  172.17.11.134 (172.17.11.134)  9.877 ms  9.775 ms  9.657 ms
 6  195.166.129.180 (195.166.129.180)  10.066 ms  9.934 ms  10.638 ms
 7  core1-BE1.southbank.ukcore.bt.net (195.99.125.130)  10.318 ms  11.101 ms  9.914 ms
 8  195.99.127.68 (195.99.127.68)  10.512 ms peer6-hu0-6-0-6.telehouse.ukcore.bt.net (195.99.127.3)  10.614 ms 109.159.252.144 (109.159.252.144)  10.382 ms
 9  213.137.183.32 (213.137.183.32)  10.835 ms 213.137.183.36 (213.137.183.36)  9.895 ms 213.137.183.32 (213.137.183.32)  10.555 ms
10  ldn-b3-link.telia.net (213.248.67.97)  10.652 ms ldn-b1-link.telia.net (213.248.97.48)  10.950 ms  11.254 ms
11  ldn-bb4-link.telia.net (62.115.116.240)  10.582 ms ldn-bb3-link.telia.net (62.115.114.234)  17.359 ms  12.718 ms
12  slou-b1-link.telia.net (62.115.117.123)  11.998 ms slou-b1-link.telia.net (62.115.117.193)  13.369 ms slou-b1-link.telia.net (62.115.117.123)  11.856 ms
13  f5defense-ic-320442-slou-b1.c.telia.net (62.115.9.217)  13.747 ms  15.239 ms  19.462 ms
14  107.162.79.5 (107.162.79.5)  11.468 ms  11.607 ms  11.928 ms
15  107.162.132.57 (107.162.132.57)  11.983 ms  11.730 ms  11.569 ms