Reachability issues for downloads.openwrt.org

_FailSafe · January 12, 2024, 8:27pm

Is there a proper place or person to contact in order to report issues with reachability to downloads.openwrt.org?

The issue I am seeing is that at my location (not sure if this is geo related), downloads.openwrt.org resolves to:

root@OpenWrt:~# dig A downloads.openwrt.org AAAA downloads.openwrt.org +short
;; Warning, extra type option
dualstack.j.sni.global.fastly.net.
2a04:4e42:4::644
dualstack.j.sni.global.fastly.net.
151.101.18.132

The IPv4 address is reachable over port 443, but the IPv6 address is not. It (IPv6) times out when attempting to access port 443.

root@OpenWrt:~# telnet 2a04:4e42:4::644 443
Trying 2a04:4e42:4::644...
telnet: connect to address 2a04:4e42:4::644: Operation timed out

This causes opkg to hang on my router, FWIW.

flygarn12 · January 12, 2024, 8:59pm

Telnet…what are you trying to do to begin with since 443 usually is HTTPS.

_FailSafe · January 12, 2024, 9:00pm

It's a troubleshooting technique.

AndrewZ · January 12, 2024, 9:01pm

I assume you have already made sure that you have an IPv6 connection.
Can you check some other addresses?

$ dig AAAA downloads.openwrt.org. +short
dualstack.j.sni.global.fastly.net.
2a04:4e42::644
2a04:4e42:200::644
2a04:4e42:400::644
2a04:4e42:600::644

I don't have an issue even with "your" address:

~# curl -I -6 --resolve downloads.openwrt.org:443:2a04:4e42:4::644 https://downloads.openwrt.org
HTTP/2 200

flygarn12 · January 12, 2024, 9:03pm

Please explain this technique how a telnet server that isn’t there are supposed to answer your telnet call?

_FailSafe · January 12, 2024, 9:08pm

Okay, I'll bite. At the risk of taking this thread way off topic...

Telnet can be used to open a connection to a remote host. It does not mean the host you are connecting to is expected to "speak telnet" back to you. This is one of many tools I, and countless other IT professionals, have used through the years to confirm if, for example, a port is open on a host or if the port may be blocked by a firewall.

One can even use the telnet command to connect directly to an smtp server to send an e-mail. Been there, done that.

So listen, I really appreciate any help with my root question. But questioning my knowledge of troubleshooting a network connection is not worth anyone's time here.

_FailSafe · January 12, 2024, 9:11pm

Thanks for the extra IPs you're seeing in your query. Out of that list, I was able to hit 2a04:4e42:400::644 one time, but then it's not responding anymore.

I tried the curl command you received the 200 with, but it works sporadically for me:

root@OpenWrt:~# curl -I -6 --resolve downloads.openwrt.org:443:2a04:4e42:400::644 https://downloads.openwrt.org
HTTP/1.1 200 OK

root@OpenWrt:~# curl -I -6 --resolve downloads.openwrt.org:443:2a04:4e42:400::644 https://downloads.openwrt.org
curl: (28) Failed to connect to downloads.openwrt.org port 443 after 130070 ms: Error

I'll do some traceroutes and see if I might be able to get some more insight. My immediate hunch is that it might be a routing/BGP issue between my ISP and Fastly.

AndrewZ · January 12, 2024, 9:33pm

Agree. Unfortunately I wasn't able to find their looking glass service.

_FailSafe · January 12, 2024, 9:39pm

I appreciate your willingness to engage with me for a bit to look into this! Traceroute confirms there is some "blackhole" going on shortly after my next hop, prior to exiting the AT&T (my ISP) network. I'll write this one off as an ISP issue at this point and give it some time to clear up.

FWIW, this is not the first time this has happened. If it occurs again, I may try to open a ticket with AT&T to see if they might indulge in looking into it. But I won't hold my breath, either.

maurer · January 13, 2024, 7:18am

When you have issues you can switch to one of the mirrors

eduperez · January 13, 2024, 8:37am

Many internet protocols (HTTP, SMTP, POP3, ...) are text-based. With a TELNET client connected to the right port (80, ...), you cat "talk" to the server and emulate a client. It was a common debugging technique, before everything went secure.

flygarn12 · January 13, 2024, 9:01am

Yea, but as I already said. 443 is TLS traffic and that port is secure but obviously I am not good enough to talk to this self claimed god?

_FailSafe · January 13, 2024, 3:07pm

Over the top, a little? I'm not a 'self claimed god'. I came to the forum here with a legitimate question and you began asking questions in such a way that insinuated I was doing something stupid by trying to use the telnet tool to establish a connection to some server on port 443.

It's an indisputable fact that what I was doing is (while perhaps "antiquated") a perfectly valid troubleshooting technique that many of us IT professionals use with regularity, and have for many years. I guess if anything, I should be humbled that maybe my years in the IT field have paid off and allowed me to reach the 'god' title.

Regardless, I got some good feedback from @AndrewZ, and I'm satisfied with where it landed. Consider it a learning success (now you know another use for telnet!) and let's move on.

flygarn12 · January 13, 2024, 5:08pm

If I remember right you actually failed this whole connection project.

And no one has still explained why a secure TLS server would accept a stupid telnet session without encryption.

lleachii · January 13, 2024, 5:14pm

From looking at this website linked below and others, a telnet test won't work on an HTTPS connection. Most websites suggest the following command:

openssl s_client -connect foo.example.com:443

See: https://www.bearfruit.org/thoughts/telnet-for-testing-ssl-https-websites/

_FailSafe · January 13, 2024, 5:41pm

It didn't fail. My tests confirmed the issue: there is some "black hole" in routing/BGP between my ISP and Fastly, specifically for IPv6.

Let me try to explain this for you guys. From my OpenWrt box, I see the following behavior for IPv4:

root@OpenWrt:~# dig A downloads.openwrt.org AAAA downloads.openwrt.org +short
;; Warning, extra type option
dualstack.j.sni.global.fastly.net.
2a04:4e42:4::644
dualstack.j.sni.global.fastly.net.
151.101.18.132

root@OpenWrt:~# telnet 151.101.18.132 443
Trying 151.101.18.132...
Connected to 151.101.18.132.
Escape character is '^]'.

Running this command in parallel confirms I have opened a TCP connection (take note of ESTABLISHED) to downloads.openwrt.org via IPv4:

root@OpenWrt:~# netstat -nap | grep telnet
tcp        0      0 99.96.XXX.XXX:33208     151.101.18.132:443      ESTABLISHED 24956/telnet

Note: With telnet I am only testing if a proper TCP handshake can be established to the host, but this confirms end-to-end connectivity.

Conversely, doing the same test with IPv6 looks like this:

root@OpenWrt:~# dig A downloads.openwrt.org AAAA downloads.openwrt.org +short
;; Warning, extra type option
dualstack.j.sni.global.fastly.net.
2a04:4e42:4::644
dualstack.j.sni.global.fastly.net.
151.101.18.132

root@OpenWrt:~# telnet 2a04:4e42:4::644 443
Trying 2a04:4e42:4::644...

But it never connects. This command in parallel shows SYN_SENT, but I never get the ACK from the IPv6 endpoint:

root@OpenWrt:~# netstat -nap | grep telnet
tcp        0      1 2001:506:70XX:XXXX::1:53764 2a04:4e42:4::644:443    SYN_SENT    3736/telnet

Now, to prove this can and does work, let's do the same with Google.com, a well-known service:

root@OpenWrt:~# dig A google.com AAAA google.com +short
;; Warning, extra type option
2607:f8b0:4004:c08::65
2607:f8b0:4004:c08::71
2607:f8b0:4004:c08::8b
2607:f8b0:4004:c08::64
172.253.63.101
172.253.63.113
172.253.63.138
172.253.63.139
172.253.63.102
172.253.63.100

root@OpenWrt:~# telnet 172.253.63.101 443
Trying 172.253.63.101...
Connected to 172.253.63.101.
Escape character is '^]'.
Connection closed by foreign host.

root@OpenWrt:~# telnet 2607:f8b0:4004:c08::65 443
Trying 2607:f8b0:4004:c08::65...
Connected to 2607:f8b0:4004:c08::65.
Escape character is '^]'.
Connection closed by foreign host.

root@OpenWrt:~# netstat -nap | grep telnet
tcp        0      0 99.96.XXX.XXX:56656     172.253.63.101:443      ESTABLISHED 8253/telnet

root@OpenWrt:~# netstat -nap | grep telnet
tcp        0      0 2600:1700:XXX:XXXX::5:34434 2607:f8b0:4004:c08::65:443 ESTABLISHED 8460/telnet

This indicates successfully negotiating a functional TCP connection to google.com via both IPv4 and IPv6. If I were to try to communicate over that established TCP connection, then that is where TLS negotiation would need to be performed next. But at a very basic level, this confirms whether a TCP connection can, or cannot, be established to a port on a remote host.

_FailSafe · January 13, 2024, 5:50pm

This deserves a deeper dive. The answer for this is simply: consider the OSI model.

My testing (the telnet test) is testing at a lower layer of the OSI model than what you're bringing up about TLS. I'm doing a layer 4 test, whereas your TLS argument is at the layer 6/7 level.

This is why telnet can confirm a completion of a handshake at the transport layer, even if I would never be able to (or want to) use telnet to communicate at layer 6/7.

Graphic borrowed from Cloudflare:

_FailSafe · January 13, 2024, 5:58pm

This is a very different testing context than I was performing yesterday. My intent for telnet was a layer 3/4 test.

system · January 23, 2024, 6:00pm

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.