Access to sysupgrade.openwrt.org is blocked

Note - I do not believe this is related to the current build queue issue as I haven't been able to reach the server at all for months.

All my traffic to sysupgrade.openwrt.org seems to be getting blocked. The name resolves to asu-02.infra.openwrt.org [45.140.183.87] for me, which I believe is correct. However nothing loads it just times out, both on my router and from any systems behind my router. This started around the time the 24.10 releases became available. A traceroute indicates traffic is getting as far as gw0.scc.kae.bb.vzffnrmo.de [45.140.183.69] and then it just gets discarded.

If I use my cell phone to open the page, or visit a friend's house who uses a different ISP, the page loads just fine, so it's my public IP that seems to be the problem. I am with a small Canadian residential ISP who has given me an address in the 45.44.69.x range and I don't know why blocking this range would be necessary. It used to work.

Can anyone contact the admins and get my address unblocked?

Thanks very much.

I got an error earlier as well. I didn't try traceroute, but I do think that this is related to the general problems facing the sysupgrade server. My guess is that the server is still having issues and that your IP is not blocked. However, I don't administer that server, so I cannot say with certainty.

When did the build problems start? I haven't been able to reach the server since at least February. I don't get an error as such I can't even ping it:

ping sysupgrade.openwrt.org

Pinging asu-02.infra.openwrt.org [45.140.183.87] with 32 bytes of data:
Request timed out.
Request timed out.
Request timed out.
Request timed out.

Ping statistics for 45.140.183.87:
    Packets: Sent = 4, Received = 0, Lost = 4 (100% loss)

Ah... I can successfully ping that address. Maybe there is indeed a block for some reason.

Hopefully @aparcar can take a look when he has some time.

Traceroute:

tracert sysupgrade.openwrt.org

Tracing route to asu-02.infra.openwrt.org [45.140.183.87]
over a maximum of 30 hops:

  1    <1 ms    <1 ms    <1 ms  [local router ip]
  2     9 ms    10 ms     9 ms  96.49.192.1
  3     *        *        *     Request timed out.
  4    13 ms    10 ms    10 ms  rc1st-be5.vc.shawcable.net [66.163.69.98]
  5    10 ms    11 ms    17 ms  177.103.44.45.in-addr.arpa [45.44.103.177]
  6    20 ms    12 ms    12 ms  81.104.44.45.in-addr.arpa [45.44.104.81]
  7    14 ms    14 ms    14 ms  van-b2-link.ip.twelve99.net [213.248.92.222]
  8    14 ms    15 ms    13 ms  sea-b1-link.ip.twelve99.net [62.115.138.38]
  9     *       59 ms    59 ms  chi-bb2-link.ip.twelve99.net [62.115.132.154]
 10     *        *        *     Request timed out.
 11   146 ms   144 ms   143 ms  ldn-bb2-link.ip.twelve99.net [62.115.139.247]
 12   150 ms   149 ms   151 ms  prs-bb2-link.ip.twelve99.net [62.115.133.239]
 13   157 ms   154 ms   157 ms  ffm-bb2-link.ip.twelve99.net [62.115.122.139]
 14   157 ms   155 ms   155 ms  ffm-b5-link.ip.twelve99.net [62.115.136.219]
 15   158 ms   164 ms   160 ms  universitt-ic-349246.ip.twelve99-cust.net [213.248.88.26]
 16   159 ms   159 ms   160 ms  stu-eti-a99-hu0-4-0-7.belwue.net [129.143.60.112]
 17   162 ms   161 ms   160 ms  kar-rz-a99-hu0-2-0-4.belwue.net [129.143.56.28]
 18     *        *        *     Request timed out.
 19     *        *        *     Request timed out.
 20   160 ms   162 ms   163 ms  gw0.scc.kae.bb.vzffnrmo.de [45.140.183.69]
 21     *        *        *     Request timed out.
 22     *        *        *     Request timed out.
 23     *        *        *     Request timed out.
 24     *        *        *     Request timed out.
 25     *        *        *     Request timed out.
 26     *        *        *     Request timed out.
 27     *        *        *     Request timed out.
 28     *        *        *     Request timed out.
 29     *        *        *     Request timed out.
 30     *        *        *     Request timed out.

Trace complete.

Keep in mind that not all hops in a traceroute will respond. But I think that the lack of a ping response to the asu server is indeed indicative of an issue... at least at first glance.

Yeah, next hop after that vzffnrmo.de should be the server:

$ tracepath 45.140.183.87
 1?: [LOCALHOST]                      pmtu 1500
 1:  _gateway                                              0.803ms
 1:  _gateway                                              0.402ms
 2:  10.72.48.1                                            9.581ms
 3:  100.120.104.122                                      10.907ms
 4:  100.120.104.18                                       10.979ms asymm  3
 5:  langbprj01-ae1.rd.la.cox.net                         12.632ms asymm  6
 6:  lax-b23-link.ip.twelve99.net                         15.076ms
 7:  ash-bb2-link.ip.twelve99.net                         71.144ms asymm  8
 8:  prs-bb1-link.ip.twelve99.net                        151.558ms asymm  9
 9:  ffm-bb1-link.ip.twelve99.net                        162.706ms asymm 15
10:  ffm-b5-link.ip.twelve99.net                         157.146ms asymm  9
11:  universitt-ic-349246.ip.twelve99-cust.net           157.621ms asymm 12
12:  stu-eti-a99-hu0-4-0-11.belwue.net                   163.884ms asymm 13
13:  kar-rz-a99-hu0-2-0-4.belwue.net                     166.924ms
14:  no reply
15:  no reply
16:  gw0.scc.kae.bb.vzffnrmo.de                          162.912ms asymm 20
17:  sysupgrade.openwrt.org                              160.324ms reached
     Resume: pmtu 1500 hops 17 back 19

@aparcar @daniel , do either of you know if there are IP blocks on asu-02.infra.openwrt.org?

Sounds like this same issue: Attended Sysupgrade Issue - #8 by the_sphynx

@the_sphynx , did it ever start working for you?

No it did not actually. Just this weekend I finally got my PBR setup so that I redirect all sysupgrade.openwrt.org destined traffic through a VPS I have a wireguard tunnel on. Works great when using the VPS' routes.

I'll try to get a fresh capture of a traceroute tomorrow and post it. Interesting others are seeing this too. I did always find it strange that somehow I was the only one experiencing this out of all of the people on the internet which also clearly indicated that data leaves my router fine. Somewhere out in the internet ether something is wrong though.

Ah now you've given me an idea. I too happen to have a WireGuard VPN set up to a friend's house. When I configure PBR to route all traffic to sysupgrade.openwrt.org over the VPN, it works!

tracert sysupgrade.openwrt.org

Tracing route to asu-02.infra.openwrt.org [45.140.183.87]
over a maximum of 30 hops:

  1    <1 ms    <1 ms    <1 ms  [local router IP]
  2    27 ms    26 ms    26 ms  [WireGuard VPN]
  3    27 ms    28 ms    27 ms  [remote router IP]
  4    36 ms    35 ms    35 ms  50.68.224.1
  5    36 ms    37 ms    34 ms  rc1st-be106-1.vc.shawcable.net [64.59.151.141]
  6    42 ms    35 ms    35 ms  24.244.60.13
  7    36 ms    41 ms    35 ms  24.83.252.66
  8     *       37 ms     *     24.83.250.1
  9    42 ms    41 ms    42 ms  rc2wt-be50-1.wa.shawcable.net [66.163.70.106]
 10    83 ms    63 ms    63 ms  rc1wt-be18-1.wa.shawcable.net [66.163.64.81]
 11     *       42 ms     *     sea-b1-link.ip.twelve99.net [213.248.67.224]
 12    84 ms    84 ms     *     chi-bb2-link.ip.twelve99.net [62.115.132.154]
 13     *        *        *     Request timed out.
 14   173 ms   171 ms   171 ms  ldn-bb2-link.ip.twelve99.net [62.115.139.247]
 15   192 ms   179 ms   174 ms  prs-bb2-link.ip.twelve99.net [62.115.133.239]
 16   191 ms   186 ms   181 ms  ffm-bb2-link.ip.twelve99.net [62.115.122.139]
 17   183 ms   182 ms   182 ms  ffm-b5-link.ip.twelve99.net [62.115.114.91]
 18   182 ms   182 ms   183 ms  universitt-ic-349246.ip.twelve99-cust.net [213.248.88.26]
 19   189 ms   190 ms   190 ms  stu-eti-a99-hu0-4-0-11.belwue.net [129.143.57.126]
 20   193 ms   195 ms   202 ms  kar-rz-a99-hu0-1-0-0.belwue.net [129.143.60.77]
 21     *        *        *     Request timed out.
 22     *        *        *     Request timed out.
 23   191 ms   193 ms   190 ms  gw0.scc.kae.bb.vzffnrmo.de [45.140.183.69]
 24   201 ms   199 ms   196 ms  sysupgrade.openwrt.org [45.140.183.87]

Trace complete.

It's an ugly hack but I'll take it :slight_smile:

One possibly relevant thing to note - it appears from this trace and my earlier trace that both of us are using shawcable.net as our ISP while theirs works and mine doesn't. This is only partially correct - in my case I am paying a 3rd party ISP who subcontracts the connection back to Shaw where as my friend buys their service from Shaw directly. I think this means I get my public IP assigned from a different block than direct customers.

Well I am afraid that the traceroute from my VPS' wireguard tunnel is less than helpful since apparently my VPS provider doesn't return traceroute info.

As for the local route using my AT&T ISP router, here is the current traceroute for that:

traceroute to sysupgrade.openwrt.org (45.140.183.87), 30 hops max, 60 byte packets
1 dsldevice.attlocal.net (192.168.1.254) 0.332 ms 0.212 ms 0.111 ms
2 107-210-168-1.lightspeed.sndgca.sbcglobal.net (107.210.168.1) 1.045 ms 1.033 ms 0.729 ms
3 71.157.16.114 (71.157.16.114) 0.723 ms 1.145 ms 1.095 ms
4 * * *
5 * * *
6 * * *
7 * * *
8 * * *
9 ash-bb2-link.ip.twelve99.net (62.115.137.38) 71.354 ms 73.822 ms rest-bb1-link.ip.twelve99.net (62.115.137.36) 71.439 ms
10 prs-bb1-link.ip.twelve99.net (62.115.140.104) 149.487 ms 146.466 ms *
11 ffm-bb1-link.ip.twelve99.net (62.115.123.12) 155.003 ms 157.088 ms 154.559 ms
12 ffm-b5-link.ip.twelve99.net (62.115.114.89) 154.427 ms 154.460 ms ffm-b5-link.ip.twelve99.net (62.115.114.91) 157.751 ms
13 universitt-ic-349246.ip.twelve99-cust.net (213.248.88.26) 161.073 ms 153.826 ms 156.684 ms
14 stu-eti-a99-hu0-4-0-11.belwue.net (129.143.57.126) 158.097 ms fra-tc-a99-hu0-1-0-0.belwue.net (129.143.56.35) 154.512 ms stu-eti-a99-hu0-4-0-11.belwue.net (129.143.57.126) 165.455 ms
15 kar-rz-a99-hu0-2-0-7.belwue.net (129.143.56.32) 164.236 ms kar-rz-a99-hu0-2-0-0.belwue.net (129.143.60.114) 158.553 ms 163.628 ms
16 * * *
17 * * *
18 gw0.scc.kae.bb.vzffnrmo.de (45.140.183.69) 162.136 ms 164.993 ms 164.033 ms
19 * * *
20 * * *
21 * * *
22 * * *
23 * * *
24 * * *
25 * * *
26 * * *
27 * * *
28 * * *
29 * * *
30 * * *

It seems quite interesting that your traceroute gets to

gw0.scc.kae.bb.vzffnrmo.de [45.140.183.69]

and then completes it's trip right to

sysupgrade.openwrt.org [45.140.183.87]

However, mine just bounces around after that same hop endlessly. Same error returned in a browser when I navigate to https://sysupgrade.openwrt.org (connection reset)

My VPS route returns pages in a browser and pings exactly as I would expect.

This is new (and very relevant) information!

Using an analysis software like tcpdump could potentially reveal the router/host sending the reset packets.

Here's curl output from a linux machine I have routing through the non-working AT&T route:

**user@pi4**:**~**$ curl -v -4 https://sysupgrade.openwrt.org
* Host sysupgrade.openwrt.org:443 was resolved.
* IPv6: (none)
* IPv4: 45.140.183.87
* Trying 45.140.183.87:443...
* connect to 45.140.183.87 port 443 from x.x.x.5 port 47436 failed: Connection timed out
* Failed to connect to sysupgrade.openwrt.org port 443 after 135450 ms: Couldn't connect to server
* Closing connection
curl: (28) Failed to connect to sysupgrade.openwrt.org port 443 after 135450 ms: Couldn't connect to server

Unfortunately the only other GUI based machine I have is an "Always on VPN" machine that won't route through that connection so I can't get a tcpdump of that particular error in the browser.

I successfully connected and updated today.

Just gave it another try and it’s still the same for me. Works via my vpn connection to another isp, but not via my local internet connection.

When mine didn't work, it didn't work even with a VPN.

Just FYI - you could have ran it on the same machine.