One site randomly not accessible on APs, wget - Failed to send request

Installing and Using OpenWrt
1

I know it's probably waste of time but maybe someone will get some idea.
I have setup described here - 802.11r - connection dropouts during handover - I'm having some specific issue with SolaxCloud connectivity - main router connects fine (testing via Wget) but those two APs are able to reach SolaxCloud just a moment after restart and then with random droupouts, sometimes permanent - when I restart network on particular AP, it works for while. I'm not aware of any other connectivity issue on those APs, just this one (using cloud services, not having any IP blocker, firewall disabled on those APs etc.). It's not issue of Solax Cloud, it is some specific problem on my APs ... routing ?

wget https://www.solaxcloud.com

tcpdump: listening on br-lan, link-type EN10MB (Ethernet), snapshot length 262144 bytes

------- WORKING FINE - tcpdump

18:41:13.926660 IP (tos 0x0, ttl 64, id 43591, offset 0, flags [DF], proto TCP (6), length 60)
    ACPRO-MK-Openwrt.lan.44928 > 47.254.152.24.443: Flags [S], cksum 0xd246 (incorrect -> 0xeaf1), seq 2070043314, win 64240, options [mss 1460,sackOK,TS val 1383315012 ecr 0,nop,wscale 4], length 0
18:41:13.939200 IP (tos 0x0, ttl 46, id 0, offset 0, flags [DF], proto TCP (6), length 60)
    47.254.152.24.443 > ACPRO-MK-Openwrt.lan.44928: Flags [S.], cksum 0x5cb3 (correct), seq 2773265377, ack 2070043315, win 14480, options [mss 1460,sackOK,TS val 1206957932 ecr 1383315012,nop,wscale 7], length 0
18:41:13.939321 IP (tos 0x0, ttl 64, id 43592, offset 0, flags [DF], proto TCP (6), length 52)
    ACPRO-MK-Openwrt.lan.44928 > 47.254.152.24.443: Flags [.], cksum 0xd23e (incorrect -> 0xb453), seq 1, ack 1, win 4015, options [nop,nop,TS val 1383315025 ecr 1206957932], length 0
18:41:13.992163 IP (tos 0x0, ttl 64, id 43593, offset 0, flags [DF], proto TCP (6), length 326)
    ACPRO-MK-Openwrt.lan.44928 > 47.254.152.24.443: Flags [P.], cksum 0xd350 (incorrect -> 0x2d70), seq 1:275, ack 1, win 4015, options [nop,nop,TS val 1383315078 ecr 1206957932], length 274
18:41:14.005507 IP (tos 0x0, ttl 46, id 7323, offset 0, flags [DF], proto TCP (6), length 52)
    47.254.152.24.443 > ACPRO-MK-Openwrt.lan.44928: Flags [.], cksum 0xc1ff (correct), seq 1, ack 275, win 122, options [nop,nop,TS val 1206957998 ecr 1383315078], length 0
18:41:14.008264 IP (tos 0x0, ttl 46, id 7324, offset 0, flags [DF], proto TCP (6), length 1500)
    47.254.152.24.443 > ACPRO-MK-Openwrt.lan.44928: Flags [.], cksum 0x2f12 (correct), seq 1:1449, ack 275, win 122, options [nop,nop,TS val 1206958000 ecr 1383315078], length 1448
18:41:14.008318 IP (tos 0x0, ttl 64, id 43594, offset 0, flags [DF], proto TCP (6), length 52)
    ACPRO-MK-Openwrt.lan.44928 > 47.254.152.24.443: Flags [.], cksum 0xd23e (incorrect -> 0xad19), seq 275, ack 1449, win 4006, options [nop,nop,TS val 1383315094 ecr 1206958000], length 0
18:41:14.008409 IP (tos 0x0, ttl 46, id 7325, offset 0, flags [DF], proto TCP (6), length 1500)
    47.254.152.24.443 > ACPRO-MK-Openwrt.lan.44928: Flags [.], cksum 0x0312 (correct), seq 1449:2897, ack 275, win 122, options [nop,nop,TS val 1206958000 ecr 1383315078], length 1448
18:41:14.008449 IP (tos 0x0, ttl 64, id 43595, offset 0, flags [DF], proto TCP (6), length 52)
    ACPRO-MK-Openwrt.lan.44928 > 47.254.152.24.443: Flags [.], cksum 0xd23e (incorrect -> 0xa791), seq 275, ack 2897, win 3974, options [nop,nop,TS val 1383315094 ecr 1206958000], length 0
18:41:14.008485 IP (tos 0x0, ttl 46, id 7326, offset 0, flags [DF], proto TCP (6), length 362)
    47.254.152.24.443 > ACPRO-MK-Openwrt.lan.44928: Flags [P.], cksum 0x8dce (correct), seq 2897:3207, ack 275, win 122, options [nop,nop,TS val 1206958000 ecr 1383315078], length 310
18:41:14.008522 IP (tos 0x0, ttl 64, id 43596, offset 0, flags [DF], proto TCP (6), length 52)
    ACPRO-MK-Openwrt.lan.44928 > 47.254.152.24.443: Flags [.], cksum 0xd23e (incorrect -> 0xa66e), seq 275, ack 3207, win 3955, options [nop,nop,TS val 1383315094 ecr 1206958000], length 0
18:41:14.153777 IP (tos 0x0, ttl 64, id 43597, offset 0, flags [DF], proto TCP (6), length 178)
    ACPRO-MK-Openwrt.lan.44928 > 47.254.152.24.443: Flags [P.], cksum 0xd2bc (incorrect -> 0x781c), seq 275:401, ack 3207, win 4006, options [nop,nop,TS val 1383315239 ecr 1206958000], length 126
18:41:14.167260 IP (tos 0x0, ttl 46, id 7327, offset 0, flags [DF], proto TCP (6), length 103)
    47.254.152.24.443 > ACPRO-MK-Openwrt.lan.44928: Flags [P.], cksum 0x19e3 (correct), seq 3207:3258, ack 401, win 122, options [nop,nop,TS val 1206958159 ecr 1383315239], length 51
18:41:14.167339 IP (tos 0x0, ttl 64, id 43598, offset 0, flags [DF], proto TCP (6), length 52)
    ACPRO-MK-Openwrt.lan.44928 > 47.254.152.24.443: Flags [.], cksum 0xd23e (incorrect -> 0xa44c), seq 401, ack 3258, win 4006, options [nop,nop,TS val 1383315253 ecr 1206958159], length 0
18:41:14.169805 IP (tos 0x0, ttl 64, id 43599, offset 0, flags [DF], proto TCP (6), length 221)
    ACPRO-MK-Openwrt.lan.44928 > 47.254.152.24.443: Flags [P.], cksum 0xd2e7 (incorrect -> 0x962c), seq 401:570, ack 3258, win 4006, options [nop,nop,TS val 1383315255 ecr 1206958159], length 169
18:41:14.185276 IP (tos 0x0, ttl 46, id 7328, offset 0, flags [DF], proto TCP (6), length 429)
    47.254.152.24.443 > ACPRO-MK-Openwrt.lan.44928: Flags [P.], cksum 0xa85d (correct), seq 3258:3635, ack 570, win 130, options [nop,nop,TS val 1206958177 ecr 1383315255], length 377
18:41:14.189867 IP (tos 0x0, ttl 64, id 43600, offset 0, flags [DF], proto TCP (6), length 52)
    ACPRO-MK-Openwrt.lan.44928 > 47.254.152.24.443: Flags [F.], cksum 0xd23e (incorrect -> 0xa201), seq 570, ack 3635, win 4006, options [nop,nop,TS val 1383315275 ecr 1206958177], length 0
18:41:14.202504 IP (tos 0x0, ttl 46, id 7329, offset 0, flags [DF], proto TCP (6), length 52)
    47.254.152.24.443 > ACPRO-MK-Openwrt.lan.44928: Flags [F.], cksum 0xb113 (correct), seq 3635, ack 571, win 130, options [nop,nop,TS val 1206958194 ecr 1383315275], length 0
18:41:14.202600 IP (tos 0x0, ttl 64, id 43601, offset 0, flags [DF], proto TCP (6), length 52)
    ACPRO-MK-Openwrt.lan.44928 > 47.254.152.24.443: Flags [.], cksum 0xd23e (incorrect -> 0xa1e2), seq 571, ack 3636, win 4006, options [nop,nop,TS val 1383315288 ecr 1206958194], length 0
18:43:51.476795 IP (tos 0x0, ttl 64, id 26132, offset 0, flags [DF], proto TCP (6), length 60)

------ NOT WORKING FEW SECONDS LATER, WGET reporting "Failed to send request: Operation not permitted" - tcpdump

ACPRO-MK-Openwrt.lan.33092 > 47.254.152.24.443: Flags [S], cksum 0xd246 (incorrect -> 0xd56e), seq 1593594727, win 64240, options [mss 1460,sackOK,TS val 1383472562 ecr 0,nop,wscale 4], length 0
18:43:52.536155 IP (tos 0x0, ttl 64, id 26133, offset 0, flags [DF], proto TCP (6), length 60)
    ACPRO-MK-Openwrt.lan.33092 > 47.254.152.24.443: Flags [S], cksum 0xd246 (incorrect -> 0xd14a), seq 1593594727, win 64240, options [mss 1460,sackOK,TS val 1383473622 ecr 0,nop,wscale 4], length 0
18:43:54.616233 IP (tos 0x0, ttl 64, id 26134, offset 0, flags [DF], proto TCP (6), length 60)
    ACPRO-MK-Openwrt.lan.33092 > 47.254.152.24.443: Flags [S], cksum 0xd246 (incorrect -> 0xc92a), seq 1593594727, win 64240, options [mss 1460,sackOK,TS val 1383475702 ecr 0,nop,wscale 4], length 0
18:43:58.696121 IP (tos 0x0, ttl 64, id 26135, offset 0, flags [DF], proto TCP (6), length 60)
    ACPRO-MK-Openwrt.lan.33092 > 47.254.152.24.443: Flags [S], cksum 0xd246 (incorrect -> 0xb93a), seq 1593594727, win 64240, options [mss 1460,sackOK,TS val 1383479782 ecr 0,nop,wscale 4], length 0
18:44:06.856104 IP (tos 0x0, ttl 64, id 26136, offset 0, flags [DF], proto TCP (6), length 60)
    ACPRO-MK-Openwrt.lan.33092 > 47.254.152.24.443: Flags [S], cksum 0xd246 (incorrect -> 0x995a), seq 1593594727, win 64240, options [mss 1460,sackOK,TS val 1383487942 ecr 0,nop,wscale 4], length 0

DNS working fine, but still WGET can't get data (and page can't be loaded on phones connected via wifi etc.)

PING www.solaxcloud.com (47.254.152.24): 56 data bytes
64 bytes from 47.254.152.24: seq=0 ttl=46 time=14.280 ms
64 bytes from 47.254.152.24: seq=1 ttl=46 time=14.052 ms

nslookup solaxcloud.com
Server:         10.0.0.1
Address:        10.0.0.1:53

Non-authoritative answer:
Name:   solaxcloud.com
Address: 47.254.152.24

Non-authoritative answer:

I have standard setup, minimal configuration...

Thank you.

{
        "kernel": "5.15.80",
        "hostname": "ACPRO_MK_Openwrt",
        "system": "Qualcomm Atheros QCA956X ver 1 rev 0",
        "model": "Ubiquiti UniFi AC Pro",
        "board_name": "ubnt,unifiac-pro",
        "rootfs_type": "squashfs",
        "release": {
                "distribution": "OpenWrt",
                "version": "SNAPSHOT",
                "revision": "r21429-57a8ea6d74",
                "target": "ath79/generic",
                "description": "OpenWrt SNAPSHOT r21429-57a8ea6d74"
        }
}
  • the same issue occurs on Ubiquiti UniFi AC Light
2

I'd start by running the latest stable release build (22..03.2).

3

Thank you, I would rather prefer a way how to diagnose this issue.
Anyway, switched to 22.03 and still facing the same.

4

ok... let's see your config files:

Please copy the output of the following commands and post it here using the "Preformatted text </> " button:
grafik
Remember to redact passwords, MAC addresses and any public IP addresses you may have:

cat /etc/config/network
cat /etc/config/wireless
cat /etc/config/dhcp
cat /etc/config/firewall
1 Like
5

As I mentioned, firewall is off, also I disabled dhcp and uninstalled odhcpd and dnsmasq (that issue I'm discussing here existed before this streamlining )...so only network and wireless might be relevant (my man router is serving as dhcp server and dns over https)

Really I don't think this is configuration issue... it's impacting one particular URL and only on connected APs, occurring randomly....

AP - cat /etc/config/network

config interface 'loopback'
        option device 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'fdce:907f:b7be::/48'

config device
        option name 'br-lan'
        option type 'bridge'
        list ports 'eth0.1'

config interface 'lan'
        option proto 'static'
        option ipaddr '10.0.0.2'
        option netmask '255.255.255.0'
        option device 'br-lan'
        list dns '10.0.0.1'
        option gateway '10.0.0.1'

config switch
        option name 'switch0'
        option reset '1'
        option enable_vlan '1'

config switch_vlan
        option device 'switch0'
        option vlan '1'
        option ports '2 3 0t'

config device
        option name 'eth0'
        option ipv6 '0'


AP - cat /etc/config/wireless

config wifi-device 'radio0'
        option type 'mac80211'
        option path 'pci0000:00/0000:00:00.0'
        option hwmode '11a'
        option noscan '1'
        option log_level '0'
        option channel '120'
        option beacon_int '75'
        option dtim_period '3'
        option htmode 'VHT80'
        option cell_density '1'
        option country 'CZ'
        option distance '20'

config wifi-device 'radio1'
        option type 'mac80211'
        option hwmode '11g'
        option path 'platform/ahb/18100000.wmac'
        option noscan '1'
        option log_level '0'
        option dtim_period '3'
        option htmode 'HT40'
        option cell_density '1'
        option beacon_int '75'
        option txpower '14'
        option channel '7'
        option country 'CZ'
        option distance '20'

config wifi-iface 'wifinet0'
        option device 'radio0'
        option mode 'ap'
        option ssid 'approwifi'
        option key '**redacted**'
        option network 'lan'
        option encryption 'psk2'
        option rrm_neighbor_report '1'
        option rrm_beacon_report '1'
        option wnm_sleep_mode '1'
        option dtim_period '3'
        option wpa_group_rekey '3600'
        option ieee80211r '1'
        option mobility_domain '4f57'
        option ft_over_ds '1'
        option ft_psk_generate_local '1'


config wifi-iface 'wifinet1'
        option device 'radio1'
        option mode 'ap'
        option ssid 'approwifi'
        option key '**redacted**'
        option network 'lan'
        option rrm_neighbor_report '1'
        option rrm_beacon_report '1'
        option wnm_sleep_mode '1'
        option encryption 'psk2'
        option wps_pushbutton '1'
        option dtim_period '1'
        option wpa_group_rekey '3600'
        option ieee80211r '1'
        option mobility_domain '4f57'
        option ft_over_ds '1'
        option ft_psk_generate_local '1'
6

I'm generally inclined to agree.

have you verified that this issue never happens on the wired network?

7

Yep, never happens, always works on wired ... it's really strange, feeling like I can't believe it myself.

Even right now I'm accessing the page from PC connected to the router via ethernet, I'm able to do WGET from the router itself, I'm NOT able to connect from phone which is connected to the AP, I'm NOT able to WGET that URL from any of the AP right now, getting ' Failed to send request: Operation not permitted' - another wireless device shows "failed, reason: connect ETIMEDOUT 47.254.152.24:443" - when I will reboot both APs, it will work for some time or few seconds, it depends.
Still I can ping from any place, also DNS works fine...

8

Your main router is at 10.0.0.1? Are there any firewall rules on the main router that may affect access to the site?

Just to make sure we're not overlooking anything, what is the IP information from your phone connected to your AP? (full info: IP, subnet mask, DNS, gateway/router)? And what about from your PC connected by ethernet?

Also, is this limited to literally only one URL? Can you ping the server where that URL is hosted?

1 Like
9

Thanks for trying to help, I appreciate. I have even tried with firewall disabled on 10.0.0.1 and anyway, there are no rules which would affect that URL - 47.254.152.24 / solaxcloud.com - still no difference.

IP 10.0.0.83, subnet 255.255.255.0, DNS/DHCP - 10.0.0.1, GW - 10.0.0.1

  Connection-specific DNS Suffix  . : lan
  Description . . . . . . . . . . . : Realtek PCIe GbE Family Controller
  Physical Address. . . . . . . . . : **redacted**
  DHCP Enabled. . . . . . . . . . . : Yes
  Autoconfiguration Enabled . . . . : Yes
  Link-local IPv6 Address . . . . . : fe80::6ea3:dbc5:ee70:a4af%4(Preferred)
  IPv4 Address. . . . . . . . . . . : 10.0.0.6(Preferred)
  Subnet Mask . . . . . . . . . . . : 255.255.255.0
  Lease Obtained. . . . . . . . . . : friday 9. December 2022 8:30:54
  Lease Expires . . . . . . . . . . : saturday 10. December 2022 3:19:41
  Default Gateway . . . . . . . . . : 10.0.0.1
  DHCP Server . . . . . . . . . . . : 10.0.0.1
  DHCPv6 IAID . . . . . . . . . . . : 57987115
  DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-23-87-**redacted**
  DNS Servers . . . . . . . . . . . : 10.0.0.1
  NetBIOS over Tcpip. . . . . . . . : Enabled

Yes, this is 47.254.152.24 / solaxcloud.com, already pasted at the beginning of this thread.

I'm now switching between different releases, seems no difference on AP side... will also upgrade router on 10.0.0.1 to see if this makes any difference.

On the above TCPDUMP, any thoughts why replies are missing for that communication or any parameters to be included for TCPDUMP analysis maybe ?

10

I'm not an expert on TCPDump, so I can't really help there.

There may be some things about the AP settings that may not be entirely optimized, but I would expect those to affect all wifi access, not selectively one site.

I don't know if you've tested this (I couldn't find an explicit mention of it, but I may have just missed it): put your PC on wifi (disconnect ethernet) and test to see if you have any dropouts. Conversely, if you have the option to put your phone on ethernet, that would be great (iPhone > lighting-to-usb adapter > USB ethernet adapter; Android > USB-C > USB-C ethernet adapter (or USB-C male-to-USB-A female > USB ethernet adapter).