jeff47
November 20, 2024, 6:35pm
1
I'm running OpenWrt 23.05.5 r24106-10cc5fcd00 / LuCI openwrt-23.05 branch git-24.313.38121-76e4eca on my Raspberry Pi 4. Upstream/WAN is through a USB-dongle.
SQM and Irqbalance are installed.
I'm having trouble keeping consistent speeds. Using Speedtest Tracker (https://github.com/alexjustesen/speedtest-tracker ), I'll get 800 Mbps after a reboot but fairly soon after, the speed drops down to 100 Mbps.
Load average on the router is 0.00/0.00/0.00. RAM is 95% free.
I need some tips on how to troubleshoot this. I suppose it could be upstream of my router, but I'm not sure why rebooting would solve that -- and it does, but only temporarily.
frollic
November 20, 2024, 7:00pm
2
temp disable SQM, see what happens ?
Let's also take a look at your config to make sure nothing sticks out as potentially problematic.
Does the speed have a continuous decline, or is there a step change to ~100Mbps?
Please connect to your OpenWrt device using ssh and copy the output of the following commands and post it here using the "Preformatted text </>
" button:
Remember to redact passwords, MAC addresses and any public IP addresses you may have:
ubus call system board
cat /etc/config/network
cat /etc/config/wireless
cat /etc/config/dhcp
cat /etc/config/firewall
jeff47
November 20, 2024, 7:06pm
4
Yeah, I thought of that... no difference at all.
jeff47
November 20, 2024, 7:20pm
5
It seems to drop straight from 800 to 100, although I can't eliminate that there is a decline between my tests.
ubus call system board
ubus call system board:
{
"kernel": "5.15.167",
"hostname": "RPi4",
"system": "ARMv8 Processor rev 3",
"model": "Raspberry Pi 4 Model B Rev 1.5",
"board_name": "raspberrypi,4-model-b",
"rootfs_type": "ext4",
"release": {
"distribution": "OpenWrt",
"version": "23.05.5",
"revision": "r24106-10cc5fcd00",
"target": "bcm27xx/bcm2711",
"description": "OpenWrt 23.05.5 r24106-10cc5fcd00"
}
}
/etc/config/network :
config interface 'loopback'
option device 'lo'
option proto 'static'
option ipaddr '127.0.0.1'
option netmask '255.0.0.0'
config globals 'globals'
option packet_steering '1'
config device
option name 'br-lan'
option type 'bridge'
option ipv6 '0'
list ports 'eth0.1'
config interface 'wan'
option proto 'dhcp'
option device 'eth1'
option delegate '0'
option peerdns '0'
list dns '192.168.1.21'
config device
option type '8021q'
option ifname 'eth0'
option vid '1'
option name 'eth0.1'
option ipv6 '0'
config device
option type '8021q'
option ifname 'eth0'
option vid '3'
option name 'eth0.3'
option ipv6 '0'
config interface 'IOT'
option proto 'static'
option device 'eth0.3'
option ipaddr '192.168.3.1'
option netmask '255.255.255.0'
option delegate '0'
list dns '192.168.1.21'
config device
option type '8021q'
option ifname 'eth0'
option vid '7'
option name 'eth0.7'
option ipv6 '0'
config interface 'CHILDSAFE'
option proto 'static'
option device 'eth0.7'
option ipaddr '192.168.7.1'
option netmask '255.255.255.0'
option delegate '0'
config interface 'LAN1'
option proto 'static'
option netmask '255.255.255.0'
option delegate '0'
option ipaddr '192.168.1.1'
option device 'br-lan'
list dns_search 'lan'
list dns '192.168.1.21'
list dns '84.200.69.80'
config device
option type '8021q'
option ifname 'eth0'
option vid '8'
option name 'eth0.8'
option ipv6 '0'
config interface 'GUEST'
option proto 'static'
option device 'eth0.8'
option ipaddr '192.168.8.1'
option netmask '255.255.255.0'
option delegate '0'
config interface 'wg0'
option proto 'wireguard'
option private_key 'xxx'
option listen_port '51281'
list addresses '192.168.254.1/24'
option delegate '0'
config wireguard_wg0 'wgclient'
option public_key 'xxx'
option preshared_key 'xxx'
list allowed_ips '192.168.254.2/32'
option private_key 'xxx'
option persistent_keepalive '25'
option description 'Jeff'\''s iPhone'
config wireguard_wg0
option public_key 'xxx'
option private_key 'xxx'
option description 'Pacific'
option preshared_key 'xxx'
list allowed_ips '192.168.254.3/32'
option persistent_keepalive '25'
config interface 'docker'
option device 'docker0'
option proto 'none'
option auto '0'
config device
option type 'bridge'
option name 'docker0'
config device
option type 'bridge'
option name 'docker1'
config interface 'dockerlan'
option proto 'none'
option device 'docker1'
option auto '1'
config interface 'awg0'
option proto 'amneziawg'
option private_key 'xxx'
option listen_port '51282'
list addresses '192.168.253.1/24'
option awg_jc '5'
option awg_jmin '50'
option awg_jmax '1000'
option awg_s1 '111'
option awg_s2 '65'
option awg_h1 '187768002'
option awg_h2 '1721857722'
option awg_h3 '319434493'
option awg_h4 '714162110'
option delegate '0'
config amneziawg_awg0
option description 'Jeff'\''s iPhone'
option public_key 'xxx'
option private_key 'xxx'
option preshared_key 'xxx'
list allowed_ips '192.168.253.2/32'
option persistent_keepalive '25'
option route_allowed_ips '1'
/etc/config/wireless (Disabled, don't use it on this device):
config wifi-device 'radio0'
option type 'mac80211'
option path 'platform/soc/fe300000.mmcnr/mmc_host/mmc1/mmc1:0001/mmc1:0001:1'
option channel '36'
option band '5g'
option htmode 'VHT80'
option disabled '1'
config wifi-iface 'default_radio0'
option device 'radio0'
option network 'lan'
option mode 'ap'
option ssid 'OpenWrt'
option encryption 'none'
/etc/config/dhcp: (static DHCPs removed)
config dnsmasq
option domainneeded '1'
option localise_queries '1'
option rebind_protection '1'
option rebind_localhost '1'
option local '/lan/'
option domain 'lan'
option expandhosts '1'
option authoritative '1'
option readethers '1'
option leasefile '/tmp/dhcp.leases'
option localservice '1'
option ednspacket_max '1232'
option dnsforwardmax '2300'
option min_cache_ttl '270'
option cachesize '5000'
list rebind_domain 'plex.direct'
list address '/router/192.168.1.1'
list address '/upstairsap/192.168.1.2'
list address '/basementap/192.168.1.3'
option localuse '1'
config odhcpd 'odhcpd'
option maindhcp '0'
option leasefile '/tmp/hosts/odhcpd'
option leasetrigger '/usr/sbin/odhcpd-update'
option loglevel '4'
/etc/config/firewall :
config defaults
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'REJECT'
option synflood_protect '1'
config zone 'lan'
option name 'lan'
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'ACCEPT'
list device 'tun0'
list device 'tun1'
list network 'LAN1'
list network 'wg0'
list network 'awg0'
config zone 'wan'
option name 'wan'
option input 'DROP'
option output 'ACCEPT'
option forward 'DROP'
option masq '1'
option mtu_fix '1'
list network 'wan6'
list network 'wan'
config forwarding
option src 'lan'
option dest 'wan'
config rule
option name 'Allow-DHCP-Renew'
option src 'wan'
option proto 'udp'
option dest_port '68'
option target 'ACCEPT'
option family 'ipv4'
config rule
option name 'Allow-Ping'
option src 'wan'
option proto 'icmp'
option icmp_type 'echo-request'
option family 'ipv4'
option target 'ACCEPT'
config rule
option name 'Allow-IGMP'
option src 'wan'
option proto 'igmp'
option family 'ipv4'
option target 'ACCEPT'
config rule
option name 'Allow-DHCPv6'
option src 'wan'
option proto 'udp'
option src_ip 'fc00::/6'
option dest_ip 'fc00::/6'
option dest_port '546'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-MLD'
option src 'wan'
option proto 'icmp'
option src_ip 'fe80::/10'
list icmp_type '130/0'
list icmp_type '131/0'
list icmp_type '132/0'
list icmp_type '143/0'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-ICMPv6-Input'
option src 'wan'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
list icmp_type 'router-solicitation'
list icmp_type 'neighbour-solicitation'
list icmp_type 'router-advertisement'
list icmp_type 'neighbour-advertisement'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-ICMPv6-Forward'
option src 'wan'
option dest '*'
option proto 'icmp'
list icmp_type 'echo-request'
list icmp_type 'echo-reply'
list icmp_type 'destination-unreachable'
list icmp_type 'packet-too-big'
list icmp_type 'time-exceeded'
list icmp_type 'bad-header'
list icmp_type 'unknown-header-type'
option limit '1000/sec'
option family 'ipv6'
option target 'ACCEPT'
config rule
option name 'Allow-IPSec-ESP'
option src 'wan'
option dest 'lan'
option proto 'esp'
option target 'ACCEPT'
config rule
option name 'Allow-ISAKMP'
option src 'wan'
option dest 'lan'
option dest_port '500'
option proto 'udp'
option target 'ACCEPT'
config rule
option name 'Allow identd'
option src 'wan'
option dest_port '113'
option target 'ACCEPT'
config include
option path '/etc/firewall.user'
option reload '1'
config redirect
option dest 'lan'
option proto 'tcp'
option target 'DNAT'
option name 'Plex'
option src 'wan'
option src_dport '32400'
option dest_ip '192.168.1.16'
config redirect
option dest 'lan'
option target 'DNAT'
option name 'Wireguard'
option src 'wan'
option dest_ip '192.168.1.16'
option src_dport '8008'
list proto 'udp'
config redirect
option dest 'lan'
option target 'DNAT'
option name 'VPN'
option src 'wan'
option dest_ip '192.168.1.16'
list proto 'tcp'
list proto 'udp'
option src_dport '8080'
config redirect
option dest 'lan'
option target 'DNAT'
option name 'SSH'
option src 'wan'
option src_dport '6114'
option dest_port '22'
option dest_ip '192.168.1.16'
list proto 'tcp'
list proto 'udp'
config redirect
option dest 'lan'
option target 'DNAT'
option name 'HTTPS'
option src 'wan'
option src_dport '443'
option proto 'tcp'
option dest_ip '192.168.1.16'
config zone
option name 'IOT'
option output 'ACCEPT'
option input 'REJECT'
option forward 'REJECT'
list network 'IOT'
config forwarding
option src 'lan'
option dest 'IOT'
config zone
option name 'CHILDSAFE'
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'ACCEPT'
list network 'CHILDSAFE'
config forwarding
option src 'CHILDSAFE'
option dest 'lan'
config forwarding
option src 'CHILDSAFE'
option dest 'wan'
config forwarding
option src 'lan'
option dest 'CHILDSAFE'
config forwarding
option src 'CHILDSAFE'
option dest 'IOT'
config include
option path '/etc/firewall.cs'
option enabled '1'
option reload '1'
config rule
option name 'Allow LAN DHCP'
option src 'lan'
option dest_port '67 68'
option target 'ACCEPT'
option family 'ipv4'
config rule
option name 'Allow IOT DHCP'
option src 'IOT'
option dest_port '53 67 68'
option target 'ACCEPT'
option family 'ipv4'
config rule
option name 'Allow IOT to Pihole'
option src 'IOT'
list dest_ip '192.168.1.21'
list dest_ip '192.168.1.22'
option dest_port '53'
option target 'ACCEPT'
option dest 'lan'
config rule
option target 'ACCEPT'
option name 'Allow IOT to ping router'
list proto 'icmp'
option src 'IOT'
list icmp_type 'echo-request'
config rule
option name 'Allow GUEST DHCP'
option src 'GUEST'
option dest_port '67 68'
option target 'ACCEPT'
option family 'ipv4'
config forwarding
option src 'IOT'
option dest 'wan'
config rule
option name 'Allow GUEST to Pihole'
option src 'GUEST'
list dest_ip '192.168.1.21'
list dest_ip '192.168.1.22'
option dest_port '53'
option target 'ACCEPT'
option dest 'lan'
config rule
option name 'Allow CHILDSAFE DHCP'
option src 'CHILDSAFE'
option dest_port '67 68'
option target 'ACCEPT'
option family 'ipv4'
config rule
option name 'Allow CHILDSAFE to Pihole'
option src 'CHILDSAFE'
list dest_ip '192.168.1.21'
list dest_ip '192.168.1.22'
option dest_port '53'
option target 'ACCEPT'
option dest 'lan'
config rule
option name 'ssh'
option src 'wan'
option target 'ACCEPT'
option src_port '22122'
option dest_port '22'
option proto 'tcp'
config rule
option name 'Allow local Plex connections'
option src '*'
option dest 'lan'
option dest_port '32400'
option target 'ACCEPT'
option proto 'tcp'
list dest_ip '192.168.1.16'
config zone
option name 'GUEST'
option output 'ACCEPT'
option forward 'REJECT'
option input 'REJECT'
list network 'GUEST'
config forwarding
option src 'GUEST'
option dest 'wan'
config include
option path '/etc/firewall.fail2ban'
option enabled '1'
option reload '1'
config redirect
option dest 'lan'
option target 'DNAT'
option name 'Router SSH'
option src 'wan'
option src_dport '1589'
option dest_ip '192.168.1.1'
option dest_port '22'
config zone 'docker'
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'REJECT'
option name 'docker'
option log '1'
list device 'docker0'
list device 'docker1'
list network 'dockerlan'
list network 'docker'
config rule
option name 'DockertoDockerAny'
option src 'docker'
option dest 'docker'
option target 'ACCEPT'
config forwarding
option src 'docker'
option dest 'wan'
config rule 'ovpn'
option name 'Allow-OpenVPN'
option src 'wan'
option dest_port '995'
option target 'ACCEPT'
list proto 'tcp'
list proto 'udp'
config redirect
option dest 'lan'
option target 'DNAT'
option name 'Vless'
option src 'wan'
option src_dport '587'
option dest_ip '192.168.1.16'
list proto 'tcp'
config redirect
option dest 'lan'
option target 'DNAT'
option name 'Trojan'
option src 'wan'
option src_dport '993'
option dest_ip '192.168.1.16'
list proto 'tcp'
config redirect
option dest 'lan'
option target 'DNAT'
option name 'Hysteria'
option src 'wan'
option src_dport '443'
option dest_ip '192.168.1.16'
list proto 'udp'
config ipset
option name 'APs'
option family 'ipv4'
list match 'src_ip'
list entry '192.168.1.3'
list entry '192.168.1.2'
config rule 'wg'
option name 'Allow-WireGuard'
option src 'wan'
option dest_port '51281'
option proto 'udp'
option target 'ACCEPT'
config rule
option name 'Allow AmneziaWG'
option src '*'
option dest_port '51282'
option target 'ACCEPT'
option family 'ipv4'
config ipset
option family 'ipv4'
option name 'local_ips'
list entry '192.168.0.0/16'
list entry '10.0.0.0/8'
list entry '172.16.0.0/12'
list entry '192.0.0.0/24'
list match 'src_net'
config ipset
option name 'piholes'
option family 'ipv4'
list match 'src_ip'
list entry '192.168.1.20'
list entry '192.168.1.21'
list entry '192.168.1.22'
list entry '192.168.1.23'
config zone
option name 'All_Local'
option input 'ACCEPT'
option output 'ACCEPT'
option forward 'REJECT'
list network 'CHILDSAFE'
list network 'GUEST'
list network 'IOT'
list network 'LAN1'
config include 'bcp38'
option type 'script'
option path '/usr/lib/bcp38/run.sh'
jeff47
November 20, 2024, 7:30pm
6
Hmm, I saw this in the kernel log:
[ 958.727547] ieee80211 phy0: brcmf_cfg80211_dump_station: BRCMF_C_GET_ASSOCLIST failed, err=-512
[ 1475.629219] r8152 2-1:1.0 eth1: carrier off
[ 1478.632481] r8152 2-1:1.0 eth1: carrier on
[ 2207.782394] r8152 2-1:1.0 eth1: carrier off
[ 2211.458818] r8152 2-1:1.0 eth1: carrier on
[ 2241.803381] r8152 2-1:1.0 eth1: carrier off
[ 2244.903812] r8152 2-1:1.0 eth1: carrier on
[ 2248.316359] r8152 2-1:1.0 eth1: carrier off
[ 2251.288719] r8152 2-1:1.0 eth1: carrier on
[ 2265.855010] r8152 2-1:1.0 eth1: carrier off
[ 2268.859328] r8152 2-1:1.0 eth1: carrier on
[ 2281.984771] r8152 2-1:1.0 eth1: carrier off
[ 2285.821710] r8152 2-1:1.0 eth1: carrier on
[ 2287.154351] r8152 2-1:1.0 eth1: carrier off
[ 2290.126317] r8152 2-1:1.0 eth1: carrier on
[ 2292.914972] r8152 2-1:1.0 eth1: carrier off
[ 2296.255164] r8152 2-1:1.0 eth1: carrier on
[ 2297.219856] r8152 2-1:1.0 eth1: carrier off
[ 2304.688236] r8152 2-1:1.0 eth1: carrier on
[ 2321.366852] r8152 2-1:1.0 eth1: carrier off
[ 2325.108894] r8152 2-1:1.0 eth1: carrier on
[ 2348.619950] r8152 2-1:1.0 eth1: carrier off
[ 2351.608056] r8152 2-1:1.0 eth1: carrier on
[ 2411.155636] r8152 2-1:1.0 eth1: carrier off
[ 2414.160448] r8152 2-1:1.0 eth1: carrier on
[ 2549.047961] r8152 2-1:1.0 eth1: carrier off
[ 2552.900392] r8152 2-1:1.0 eth1: carrier on
[ 2587.373642] r8152 2-1:1.0 eth1: carrier off
[ 2594.842575] r8152 2-1:1.0 eth1: carrier on
[ 2599.935280] r8152 2-1:1.0 eth1: carrier off
[ 2602.925870] r8152 2-1:1.0 eth1: carrier on
[ 2605.552059] r8152 2-1:1.0 eth1: carrier off
[ 2608.540516] r8152 2-1:1.0 eth1: carrier on
[ 2627.395310] r8152 2-1:1.0 eth1: carrier off
[ 2631.104542] r8152 2-1:1.0 eth1: carrier on
[ 2632.500525] r8152 2-1:1.0 eth1: carrier off
[ 2635.473122] r8152 2-1:1.0 eth1: carrier on
[ 2636.853063] r8152 2-1:1.0 eth1: carrier off
[ 2639.859136] r8152 2-1:1.0 eth1: carrier on
[ 2641.319979] r8152 2-1:1.0 eth1: carrier off
[ 2644.258456] r8152 2-1:1.0 eth1: carrier on
[ 2659.656025] r8152 2-1:1.0 eth1: carrier off
[ 2662.628244] r8152 2-1:1.0 eth1: carrier on
[ 2676.203637] r8152 2-1:1.0 eth1: carrier off
[ 2679.207571] r8152 2-1:1.0 eth1: carrier on
[ 2683.036794] r8152 2-1:1.0 eth1: carrier off
[ 2691.177353] r8152 2-1:1.0 eth1: carrier on
[ 2692.397213] r8152 2-1:1.0 eth1: carrier off
[ 2699.802392] r8152 2-1:1.0 eth1: carrier on
[ 2712.527808] r8152 2-1:1.0 eth1: carrier off
[ 2727.373780] r8152 2-1:1.0 eth1: carrier on
[ 3789.555249] hrtimer: interrupt took 4759 ns
[11652.175206] ieee80211 phy0: brcmf_cfg80211_dump_station: BRCMF_C_GET_ASSOCLIST failed, err=-512
I wonder if my usb ethernet dongle is glitching?
frollic
November 20, 2024, 7:31pm
7
are you measuring this over wifi ?
that's always a bad idea.
There is a lot of stuff that doesn't look right here.
My recommendation would be to use a basic config and see if the problem manifests. The easiest and least invasive way to do this would be to simply install OpenWrt on another microSD card and insert that. Do the minimum required configuration and then use it (no VPNs, no VLANs, etc.).
If you still see the problem, you should start by looking at your cables/connections. It's possible you have a faulty cable or port that is dropping to 100Mbps. Even new and/or "high spec" cables can be bad, so don't make any assumptions. Also check the ports/connectors for any debris or bent pins.
T
1 Like
jeff47
November 20, 2024, 7:33pm
9
No, measurement is over ethernet.
frollic
November 20, 2024, 7:34pm
10
then why are you bringing up the usb wifi adapter ?
if you think it could be related, pull it.
jeff47
November 20, 2024, 7:36pm
11
Because I wrote the wrong thing. It's a usb ethernet adapter to my ISP provider, and the built-in ethernet to my LAN.
I should point out that this general setup has been stable at 800 Mbps for months, and it's not much different from similar setups using the Rpi. But something recently has gone wrong...
Just not sure where the issue is that it works at high speed for a little while, and then drops... seems like I should be able to see something clogging up the works in that case.
A faulty cable or port (or a piece of kit that is beginning to fail) could well cause the issue you're seeing... so it may not be directly related to your configuration. But to rule in/out config issues (because as I said, there's a bunch of stuff that doesn't look right), I strongly recommend using a minimally configured setup so you can identify a root cause as likely hardware vs likely configuration based.
jeff47
November 20, 2024, 7:42pm
13
psherman:
A faulty cable or port (or a piece of kit that is beginning to fail) could well cause the issue you're seeing... so it may not be directly related to your configuration. But to rule in/out config issues (because as I said, there's a bunch of stuff that doesn't look right), I strongly recommend using a minimally configured setup so you can identify a root cause as likely hardware vs likely configuration based.
Thanks, will try this and report back. I have a spare RPi and ethernet adapter, so I can try this same config on different hardware.
frollic
November 20, 2024, 7:44pm
14
I'd try the same everything, but the config
It may not be the Pi that is at fault... I don't recommend using a different Pi with the same config.
Instead, use the same pi as you're currently using with a bare-bones config.
If the problem manifests at that point, it could be hardware...
so then you can change out the Pi (maintaining the minimalist config).
If it still happens after changing out the Pi, it's a problem not related to the Pi, but maybe another piece of hardware.
If it resolves, when you change out the Pi, it's the Pi or ethernet dongle that is the problem
if it resolves when you change out the config, it's the config that is the problem.
jeff47:
Hmm, I saw this in the kernel log:
[ 1475.629219] r8152 2-1:1.0 eth1: carrier off
[ 1478.632481] r8152 2-1:1.0 eth1: carrier on
[ 2207.782394] r8152 2-1:1.0 eth1: carrier off
[ 2211.458818] r8152 2-1:1.0 eth1: carrier on
[ 2241.803381] r8152 2-1:1.0 eth1: carrier off
[ 2244.903812] r8152 2-1:1.0 eth1: carrier on
My (admittedly random) guess would be an autonegotiation problem on the link.
Try fixing the port to 1000mbps + full-duplex ?
Example for the RPi end of the link: ethtool -s eth1 duplex full speed 1000
Do the same thing on the other end and see if it helps.
With the same link partner (equipment on the other end)?
No, do not set a fixed speed. This will end up causing more problems than it will solve. When you set a port to fixed gigabit, it will still technically go through an auto-negotiation process, but with the minimum negotiation value of 1000Mbps/Full duplex. This will cause it to come up (if possible because of an intermittent/marginal issue), or to fail entirely because it can't drop down to 100Mbps. But, that doesn't help identify the root cause.
As I said before, check cables, terminations, and ports.
If setting a fixed speed causes the link to go down entirely.. Surely that would in fact help identify the root cause as being related to link speed autonegotiation?
well, yes, but with a complete downing of the network (rather than just a degradation of performance). That makes it go from a nuisance to critical failure. And t still doesn't help you identify the physical culprit (cable, terminations, ports, or failing hardware).
jeff47
November 20, 2024, 9:14pm
20
Upstream is my ISP... so it's a bit of a blackbox.