Inconsistent oddities, can't make sense of this

Installing and Using OpenWrt
1

Okay, Ive really really tried to avoid a post, but going on months now and I've exhausted my abilities. Even gave a few AIs a crack at this, but can't make sense of what's happening.

I'm running an RM551e in ECM mode via PCIe (passthrough mode, so 192.0.0.2 address being assigned) running a 'custom' openwrt into a bpi-r3 I just upgraded to 24.10-RC5 yesterday. Tried different ports, different cables, different switches, it makes no sense..

The issue? The modem directly to the desktop PC flies. Easily clears 2gbps down, 300mbps up.

The RM520F did not run openwrt, but connected to the same router would provide similar speeds on router and to desktop..

Anyways, the symptoms.. On the router, speedtest is only 500/20.. On some desktops, 2000/10.. On others, 20/50.. Even on the same switch.

The one that really gets me? I have my jellyfin on a tailscale tunnel that easily exceeds 300mbps upload without issue (when it decides to direct connect without a DERP, which also randomly won't, despite other devices on the network getting direct connections without issue).. The only device on the entire network capable of consistently going over 100mbps up.. So tailscale must be doing something, right? Well... One port over on the same switch (have changed the switch, the cable, and the network adapter with no success here), my other server that connects to my VPS through tailscale as well can't get over 10mbps... Same method, same tailnet, same network.. Completely different results.

Sometimes I can connect to PIA just fine, sometimes can't. Sometimes wireguard doesnt work and sometimes openvpn doesnt work.. On the same device. Just depends on how it feels at any given moment.

I can't find a pattern, and Ive been trying for months now. The completely fresh install yesterday was an attempt to start over, but the issue persists.

Iperf3 --bidir is almost always great between all of these devices (when Im not messing with the mtu, at least), even device to modem, usually getting over 2gbps each way without issue.

As i said, direct from the modem to the desktop, it absolutely flies, 2gbps+ down, 300mbps up without issue. Same desktop can clear 2gbps down through the router, but only 5-10mbps up. Which is on the same switch as the jellyfin server that can do 300mbps..

What gives?! Can anyone at least point me in a different direction? Ive tried mss clamping/mtu adjusting so many times. LAN masquerading seemed to help somewhat initially, but now it's not? Im going in circles here. Flow control makes no difference. Wide open firewall made no difference. You'd think the router itself would have decent speeds, but even thats getting weirdly slow speeds in both directions.. So i can get 2gbps down on some devices, but the router itself can never seem to clear 500mbps down.

Im certain the modem is likely doing something odd (and Im afraid to touch it because it's outdoors and requires a ladder/screwdriver to get to, and we're in a brutal cold snap right now, no connection other than via the network, so if i break that it'll be a very bad time) but that i can effortlessly achieve full speed on desktop makes me think Ive gotta be missing something on the bpi-r3's 24.10-RC5 install. Even if its not the "right" fix to work around whatever oddity the modem is doing, thats still better than this.

I can provide any configs or outputs requested from either of these devices... But clearly something is extremely weird here.. Happy to provide additional info or anything that can help you to help me.

I truly appreciate any guidance into diagnosing this odd issue. Thanks for reading.

2

I think the best approach is to start by looking at your complete config. We may spot some issues there.

Please connect to your OpenWrt device using ssh and copy the output of the following commands and post it here using the "Preformatted text </> " button:
grafik
Remember to redact passwords, MAC addresses and any public IP addresses you may have:

ubus call system board
cat /etc/config/network
cat /etc/config/wireless
cat /etc/config/dhcp
cat /etc/config/firewall
3

Thank you so much for the response! Here's what i can provide, sorry, my head was swimming and this should've been obvious.

I'll include the modem since its... A mess.. And you'll see why I suspect something here..

From the modem:

root@OpenWrt:~# ubus call system board
{
        "kernel": "5.15.137-debug",
        "hostname": "OpenWrt",
        "system": "ARMv8 Processor rev 0",
        "model": "Qualcomm Technologies, Inc. SDXPINN IDP M2",
        "board_name": "qcom-sdxpinn-idp",
        "rootfs_type": "ubifs",
        "release": {
                "distribution": "OpenWrt",
                "version": "22.03.5",
                "revision": "r20134-5f15225c1e",
                "target": "sdx75/generic",
                "description": "OpenWrt 22.03.5 r20134-5f15225c1e"
        }
}
root@OpenWrt:~# cat /etc/config/network

config interface 'loopback'
        option device 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option packet_steering '1'

config interface 'lan'
        option device 'br-lan'
        option proto 'static'
        option ipaddr '192.168.224.1'
        option netmask '255.255.252.0'

config interface 'rndis'
        option device 'rndis0'
        option proto 'static'
        option ipaddr '169.254.20.1'
        option netmask '255.255.255.0'

config interface 'ecm'
        option device 'ecm0'
        option proto 'static'
        option ipaddr '169.254.21.1'
        option netmask '255.255.255.0'

config interface 'eth'
        option device 'eth0'
        option proto 'static'
        option ipaddr '169.254.22.1'
        option netmask '255.255.255.0'

config interface 'eth_nic2'
        option device 'eth1'
        option proto 'static'
        option ipaddr '169.254.23.1'
        option netmask '255.255.255.0'

config interface 'eth_usb'
        option device 'eth2'
        option proto 'static'
        option ipaddr '169.254.24.1'
        option netmask '255.255.255.0'

config interface 'wan'
        option proto 'rmnet'
        option profile '1'
        option downstream 'lan'
        option type 'internet'
        option bind '0'
        option ipv6 '0'
        option lockmtu '1'
        option mtu '1472'
        option defaultroute '1'

config interface 'wan_v6'
        option proto 'rmnet'
        option profile '1'
        option downstream 'lan'
        option type 'internet'
        option bind '0'
        option ipv6 '1'
        option lockmtu '1'
        option mtu '1500'
        option defaultroute '1'

config interface 'waneth'
        option defaultroute '0'
        option proto 'dhcpv6'

config interface 'waneth_v6'
        option proto 'dhcpv6'
        option ipv6 '1'
        option type 'internet'
        option downstream 'lan'
        option defaultroute '0'
        option lockmtu '1'

config interface 'wanusb'
        option proto 'dhcp'
        option defaultroute '0'

config interface 'wanusb_v6'
        option proto 'dhcpv6'
        option ipv6 '1'
        option type 'internet'
        option downstream 'lan'
        option defaultroute '0'
        option lockmtu '1'

config device
        option name 'br-lan'
        option type 'bridge'
        list ports 'ecm0'
        list ports 'eth0'
        list ports 'eth1'
        list ports 'eth2'
        list ports 'rndis0'
        option mtu '1500'
        option mtu6 '1500'

config interface 'wan2'
        option proto 'rmnet'
        option profile '2'
        option type 'internet'
        option bind '1'
        option ipv6 '0'
        option defaultroute '0'

config interface 'wan2_v6'
        option proto 'rmnet'
        option profile '2'
        option type 'internet'
        option bind '1'
        option ipv6 '1'
        option defaultroute '0'

config interface 'wan3'
        option proto 'rmnet'
        option profile '3'
        option type 'internet'
        option bind '1'
        option ipv6 '0'
        option defaultroute '0'

config interface 'wan3_v6'
        option proto 'rmnet'
        option profile '3'
        option type 'internet'
        option bind '1'
        option ipv6 '1'
        option defaultroute '0'

config interface 'wan4'
        option proto 'rmnet'
        option profile '4'
        option type 'internet'
        option bind '1'
        option ipv6 '0'
        option defaultroute '0'

config interface 'wan4_v6'
        option proto 'rmnet'
        option profile '4'
        option type 'internet'
        option bind '1'
        option ipv6 '1'
        option defaultroute '0'

config interface 'lan_bind4'
        option device 'br-lan'
        option proto 'static'
        option ipaddr '192.0.0.1'
        option netmask '255.255.255.255'

config interface 'lan_bind6'
        option device 'br-lan'
        option proto 'static'
        option ip6addr '2607:*******::1/64'

root@OpenWrt:~# cat /etc/config/dhcp

config dhcp 'lan'
        option interface 'lan'
        option start '100'
        option leasetime '12h'
        option dhcpv4 'server'
        option dhcpv6 'server'
        option ra 'server'
        option ra_slaac '1'
        list ra_flags 'managed-config'
        list ra_flags 'other-config'
        option limit '768'
        option instance 'lan_dns'
        option ignore '0'
        list dhcp_option_force '26,1500'
        list dhcp_option_force '26,1500'
        list dhcp_option_force '26,1500'
        list dhcp_option_force '26,1500'
        list dhcp_option_force '26,1472'
        list dns 'fd00:976a::9'
        list dns 'fd00:976a::10'
        list dns 'fd00:976a::9'
        list dns 'fd00:976a::10'
        option ra_mtu '1500'
        option preferred_lifetime '28800'
        option ra_useleasetime '1'

config dhcp 'wan'
        option interface 'wan'
        option ignore '1'

config odhcpd 'odhcpd'
        option maindhcp '0'
        option leasefile '/tmp/hosts/odhcpd'
        option leasetrigger '/usr/sbin/odhcpd-update'
        option loglevel '4'
        option dad_forward '1'

config dnsmasq 'lan_dns'
        option domainneeded '1'
        option boguspriv '1'
        option filterwin2k '0'
        option localise_queries '1'
        option rebind_protection '1'
        option rebind_localhost '1'
        option local '/lan/'
        option domain 'lan'
        option expandhosts '1'
        option nonegcache '0'
        option authoritative '1'
        option readethers '1'
        option leasefile '/tmp/data/dhcp.leases.lan'
        option resolvfile '/tmp/resolv.conf.d/resolv.conf.lan.auto'
        option nonwildcard '1'
        option localservice '1'
        option ednspacket_max '1232'
        list interface 'lan'
        list notinterface 'loopback'
        option dhcpscript '/etc/data/dnsmasq_script.sh'

config dhcp 'wanbt_v6'
        option interface 'wanbt_v6'

config dhcp 'waneth_v6'
        option interface 'waneth_v6'

config dhcp 'wanusb_v6'
        option interface 'wanusb_v6'

config dhcp 'wanwlan_v6'
        option interface 'wanwlan_v6'

config dhcp 'wan2'
        option interface 'wan2'
        option ignore '1'

config dhcp 'wan3'
        option interface 'wan3'
        option ignore '1'

config dhcp 'wan4'
        option interface 'wan4'
        option ignore '1'

config dhcp 'lan_bind4'
        option leasetime '12h'
        option start '192.0.0.2'
        option dhcpv4 'server'
        option interface 'lan_bind4'
        option limit '1'
        option netmask '255.255.255.224'
        option instance 'lan_dns'
        list dhcp_option_force '26,1472'
        option ippt_bind4 '1'
        list dhcp_option '6,192.168.224.1'

config host
        option dns '1'
        option ip '192.0.0.2'
        option mac 'de:ad:be:ef:ca:fa'
        option instance 'lan_dns'
        option ippt_host '1'

root@OpenWrt:~# cat /etc/config/firewall

config defaults
        option input 'ACCEPT'
        option output 'ACCEPT'
        option flow_offloading '0'
        option forward 'ACCEPT'
        option synflood_protect '1'
        option drop_invalid '1'

config forwarding
        option dest 'wan'
        option src 'lan_wan'

config include
        option path '/etc/firewall.user'

config zone
        option name 'lan_wan'
        list network 'lan'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'ACCEPT'
        option mtu_fix '1'
        option masq '1'

config zone
        option name 'lan'
        list network 'lan'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'ACCEPT'
        option mtu_fix '1'

config zone
        option name 'wan_all'
        option input 'REJECT'
        option output 'ACCEPT'
        option mtu_fix '1'
        option forward 'ACCEPT'
        list network 'wan_v6'
        list network 'wan'

config zone
        option name 'wan'
        option output 'ACCEPT'
        option mtu_fix '1'
        list network 'waneth'
        list network 'waneth_v6'
        list network 'wan_v6'
        list network 'wan'
        option forward 'ACCEPT'
        option input 'ACCEPT'

config rule
        option name 'Allow-ICMP-Input'
        option src 'wan_all'
        option proto 'icmp'
        option family 'ipv4'
        option target 'ACCEPT'
        option enabled '1'

config rule
        option name 'Allow-ICMPv6-Input'
        option src 'wan_all'
        option proto 'icmp'
        option family 'ipv6'
        option target 'ACCEPT'
        option enabled '1'

config rule
        option name 'Allow-ICMPv6-Forward'
        option src 'wan_all'
        option dest '*'
        option proto 'icmp'
        option family 'ipv6'
        option target 'ACCEPT'
        option enabled '0'

config rule
        option name 'Allow-TCP-Input'
        option src 'wan_all'
        option proto 'tcp'
        option family 'ipv4'
        option target 'ACCEPT'
        option enabled '1'

config rule
        option name 'Allow-TCP-Forward'
        option src 'wan_all'
        option dest '*'
        option proto 'tcp'
        option family 'ipv4'
        option target 'ACCEPT'
        option enabled '1'

config rule
        option name 'Allow-TCP-v6-Input'
        option src 'wan_all'
        option proto 'tcp'
        option family 'ipv6'
        option target 'ACCEPT'
        option enabled '1'

config rule
        option name 'Allow-TCP-v6-Forward'
        option src 'wan_all'
        option dest '*'
        option proto 'tcp'
        option family 'ipv6'
        option target 'ACCEPT'
        option enabled '1'

config rule
        option name 'Allow-UDP-Input'
        option src 'wan_all'
        option proto 'udp'
        option family 'ipv4'
        option target 'ACCEPT'
        option enabled '1'

config rule
        option name 'Allow-UDP-Forward'
        option src 'wan_all'
        option dest '*'
        option proto 'udp'
        option family 'ipv4'
        option target 'ACCEPT'
        option enabled '1'

config rule
        option name 'Allow-UDP-V6-Input'
        option src 'wan_all'
        option proto 'udp'
        option family 'ipv6'
        option target 'ACCEPT'
        option enabled '1'

config zone
        option name 'wan2'
        option network 'wan2'
        option output 'ACCEPT'
        option masq '1'
        option mtu_fix '1'
        option forward 'ACCEPT'
        option input 'ACCEPT'

config zone
        option name 'lan_wan2'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'ACCEPT'
        option mtu_fix '1'
        option masq '1'

config forwarding
        option src 'lan_wan2'
        option dest 'wan2'

config zone
        option name 'wan3'
        option network 'wan3'
        option output 'ACCEPT'
        option masq '1'
        option mtu_fix '1'
        option forward 'ACCEPT'
        option input 'ACCEPT'

config zone
        option name 'lan_wan3'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'ACCEPT'
        option mtu_fix '1'
        option masq '1'

config forwarding
        option src 'lan_wan3'
        option dest 'wan3'

config zone
        option name 'wan4'
        option network 'wan4'
        option output 'ACCEPT'
        option masq '1'
        option mtu_fix '1'
        option forward 'ACCEPT'
        option input 'ACCEPT'

config zone
        option name 'lan_wan4'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'ACCEPT'
        option mtu_fix '1'
        option masq '1'

config forwarding
        option src 'lan_wan4'
        option dest 'wan4'

config rule
        option src 'lan'
        option dest '*'
        option proto 'udp'
        option family 'ipv4'
        option target 'MARK'
        option set_mark '0xc8'
        option dest_port '67'
        option enabled '1'

config rule
        option name 'IPPT-lan-Mangle-PREROUTING'
        option src 'lan'
        option dest '*'
        option proto 'udp'
        option family 'ipv4'
        option target 'MARK'
        option set_mark '0xc8'
        option dest_port '67'
        option enabled '1'

config rule
        option name 'IPPT-lan-Mangle-OUTPUT'
        option dest 'lan'
        option proto 'udp'
        option family 'ipv4'
        option target 'MARK'
        option set_mark '0xc8'
        option dest_port '67'
        option enabled '1'

config redirect
        option dest 'lan'
        option target 'DNAT'
        option name 'Poo'
        list proto 'all'
        option src 'wan'
        option dest_ip '192.0.0.2'

The bpi-r3 (the "main" router)

root@OpenWrt:~# ubus call system board
{
        "kernel": "6.6.69",
        "hostname": "OpenWrt",
        "system": "ARMv8 Processor rev 4",
        "model": "Bananapi BPI-R3",
        "board_name": "bananapi,bpi-r3",
        "rootfs_type": "squashfs",
        "release": {
                "distribution": "OpenWrt",
                "version": "24.10.0-rc5",
                "revision": "r28304-6dacba30a7",
                "target": "mediatek/filogic",
                "description": "OpenWrt 24.10.0-rc5 r28304-6dacba30a7",
                "builddate": "1736026537"
        }
}
root@OpenWrt:~# cat /etc/config/network

config interface 'loopback'
        option device 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'fd##:57##:2e##::/48'

config device
        option name 'br-lan'
        option type 'bridge'
        list ports 'lan1'
        list ports 'lan2'
        list ports 'lan3'
        list ports 'lan4'
        list ports 'sfp2'

config interface 'lan'
        option device 'br-lan'
        option proto 'static'
        option ipaddr '192.168.1.1'
        option netmask '255.255.255.0'
        option ip6assign '60'

config device
        option name 'br-wan'
        option type 'bridge'
        list ports 'eth1'
        list ports 'wan'

config device
        option name 'eth1'
        option macaddr '42:47:8c:##:##:##'

config device
        option name 'wan'
        option macaddr '42:47:8c:8c:##:##'

config interface 'wan'
        option device 'br-wan'
        option proto 'dhcp'

config interface 'wan6'
        option device 'br-wan'
        option proto 'dhcpv6'

root@OpenWrt:~# cat /etc/config/wireless

config wifi-device 'radio0'
        option type 'mac80211'
        option path 'platform/soc/18000000.wifi'
        option band '2g'
        option channel '1'
        option htmode 'HE40'
        option txpower '12'
        option country 'PA'
        option cell_density '0'
        option noscan '1'

config wifi-device 'radio1'
        option type 'mac80211'
        option path 'platform/soc/18000000.wifi+1'
        option band '5g'
        option channel '36'
        option htmode 'HE160'
        option txpower '25'
        option country 'PA'
        option cell_density '0'

config wifi-iface 'wifinet2'
        option device 'radio0'
        option mode 'ap'
        option ssid 'APname'
        option encryption 'psk2+ccmp'
        option key 'password'
        option ieee80211r '1'
        option mobility_domain 'F0CC'
        option reassociation_deadline '20000'
        option ft_over_ds '0'
        option ft_psk_generate_local '1'
        option network 'lan'

config wifi-iface 'wifinet3'
        option device 'radio1'
        option mode 'ap'
        option ssid 'APname'
        option encryption 'psk2+ccmp'
        option ieee80211r '1'
        option mobility_domain 'F0CC'
        option reassociation_deadline '20000'
        option ft_over_ds '0'
        option ft_psk_generate_local '1'
        option network 'lan'
        option key 'password'

config wifi-iface 'wifinet4'
        option device 'radio1'
        option mode 'ap'
        option ssid 'virtualAP'
        option encryption 'psk2+ccmp'
        option key 'anotherpassword'
        option network 'lan'

root@OpenWrt:~# cat /etc/config/dhcp

config dnsmasq
        option domainneeded '1'
        option localise_queries '1'
        option rebind_protection '1'
        option rebind_localhost '1'
        option local '/lan/'
        option domain 'lan'
        option expandhosts '1'
        option cachesize '1000'
        option authoritative '1'
        option readethers '1'
        option leasefile '/tmp/dhcp.leases'
        option resolvfile '/tmp/resolv.conf.d/resolv.conf.auto'
        option localservice '1'
        option ednspacket_max '1232'
        option confdir '/tmp/dnsmasq.d'

config dhcp 'lan'
        option interface 'lan'
        option start '100'
        option limit '150'
        option leasetime '12h'
        option dhcpv4 'server'
        option dhcpv6 'server'
        option ra 'server'
        option ra_slaac '1'
        list ra_flags 'managed-config'
        list ra_flags 'other-config'

config dhcp 'wan'
        option interface 'wan'
        option ignore '1'

config odhcpd 'odhcpd'
        option maindhcp '0'
        option leasefile '/tmp/hosts/odhcpd'
        option leasetrigger '/usr/sbin/odhcpd-update'
        option loglevel '4'

config host
        option name 'blumpkin'
        option dns '1'
        option ip '192.168.1.23'
        list mac 'F8:B1:56:##:##:##'
        list mac '6C:1F:F7:##:##:##'

config host
        option dns '1'
        option ip '192.168.1.33'
        option name 'hdhr-e'
        list mac '00:18:DD:##:##:##'

config host
        option name 'hdhr-s'
        option dns '1'
        option ip '192.168.1.31'
        list mac '00:18:DD:##:##:##'

config host
        option name 'hdhr-w'
        option dns '1'
        option ip '192.168.1.32'
        list mac '00:18:DD:##:##:##'

config host
        option name 'windstream'
        option dns '1'
        option mac 'E8:6F:F2:##:##:##'
        option ip '192.168.1.10'

config host
        option name 'tplink'
        option dns '1'
        option mac 'AC:84:c6:##:##:##'
        option ip '192.168.1.3'

config host
        option name 'asus'
        option dns '1'
        option mac '34:97:F6:##:##:##'
        option ip '192.168.1.2'

config host
        option name 'kodi'
        option dns '1'
        option mac '00:1E:06:##:##:##'
        option ip '192.168.1.111'

config host
        option name 'mcnips'
        option dns '1'
        option ip '192.168.1.234'
        list mac 'B0:6E:BF:##:##:##'
        list mac '10:7C:61:##:##:##'

config host
        option mac 'CC:F7:35:##:##:##'
        option name 'firetv'
        option dns '1'
        option ip '192.168.1.252'

config host
        option mac '00:04:4B:##:##:##'
        option name 'Shield'
        option dns '1'
        option ip '192.168.1.209'

config host
        option mac 'F4:4D:30:##:##:##'
        option name 'shooter-mini'
        option dns '1'
        option ip '192.168.1.202'

config host
        option name 'yamaha'
        option dns '1'
        option mac 'AC:44:F2:##:##:##'
        option ip '192.168.1.175'

config host
        option mac '2C:F0:5D:##:##:##'
        option name 'guns-desktop'
        option dns '1'
        option ip '192.168.1.216'

config host
        option name 'guns-phone'
        option dns '1'
        option ip '192.168.1.187'

config domain
        option name 'music'
        option ip '192.168.1.23'

config domain
        option ip '192.168.1.23'
        option name 'Music.blumpkin'

config domain
        option name 'broadcast'
        option ip '192.168.1.23'

config host
        option mac 'F4:CE:46:##:##:##'
        option name 'wnga'
        option dns '1'
        option ip '192.168.1.96'

config domain
        option ip '192.168.1.23'
        option name 'music.rags.gq'

config host
        option name 'ax1500'
        option dns '1'
        option mac '44:56:E2:##:##:##'
        option ip '192.168.1.5'

config host
        option name 'token-linux'
        list mac 'B0:6E:BF:##:##:##'

config host
        option name 'birbcam'
        option mac '34:3E:A4:##:##:##'
        option dns '1'
        option ip '192.168.1.71'
        option tag 'custom_gtwy'

config tag 'custom_gtwy'
        list dhcp_option '3,192.168.1.10'

config host
        option name 'unraid'
        option dns '1'
        option mac 'F4:B5:20:##:##:##'
        option ip '192.168.1.193'

config host
        option name 'P25'
        option dns '1'
        option mac 'B8:27:EB:##:##:##'
        option ip '192.168.1.25'

config domain
        option name 'ai.blumpkin.lan'
        option ip '192.168.1.23'

config domain
        option name 'jelly.lan'
        option ip '192.168.1.23'

config domain
        option name 'rooter'
        option ip '192.168.1.1'

config domain
        option name 'jelly.domain.org'
        option ip '192.168.1.23'

config dhcp 'wan6'
        option interface 'wan6'
        option ignore '1'

config domain
        option name 'homeassistant.local'
        option ip '192.168.1.23'

config host
        option name 'tokenfoan'
        list mac '98:09:CF:##:##:##'
        option ip '192.168.1.233'
        option leasetime 'infinite'

root@OpenWrt:~# cat /etc/config/firewall

config defaults
        option input 'REJECT'
        option output 'ACCEPT'
        option forward 'REJECT'
        option synflood_protect '1'
        option flow_offloading '1'
        option flow_offloading_hw '1'

config zone
        option name 'lan'
        list network 'lan'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'ACCEPT'
        option masq '1'
        option mtu_fix '1'

config zone
        option name 'wan'
        list network 'wan'
        list network 'wan6'
        option input 'REJECT'
        option output 'ACCEPT'
        option forward 'REJECT'
        option masq '1'
        option mtu_fix '1'

config forwarding
        option src 'lan'
        option dest 'wan'

config rule
        option name 'Allow-DHCP-Renew'
        option src 'wan'
        option proto 'udp'
        option dest_port '68'
        option target 'ACCEPT'
        option family 'ipv4'

config rule
        option name 'Allow-Ping'
        option src 'wan'
        option proto 'icmp'
        option icmp_type 'echo-request'
        option family 'ipv4'
        option target 'ACCEPT'

config rule
        option name 'Allow-IGMP'
        option src 'wan'
        option proto 'igmp'
        option family 'ipv4'
        option target 'ACCEPT'

config rule
        option name 'Allow-DHCPv6'
        option src 'wan'
        option proto 'udp'
        option dest_port '546'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-MLD'
        option src 'wan'
        option proto 'icmp'
        option src_ip 'fe80::/10'
        list icmp_type '130/0'
        list icmp_type '131/0'
        list icmp_type '132/0'
        list icmp_type '143/0'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-ICMPv6-Input'
        option src 'wan'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        list icmp_type 'router-solicitation'
        list icmp_type 'neighbour-solicitation'
        list icmp_type 'router-advertisement'
        list icmp_type 'neighbour-advertisement'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-ICMPv6-Forward'
        option src 'wan'
        option dest '*'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-IPSec-ESP'
        option src 'wan'
        option dest 'lan'
        option proto 'esp'
        option target 'ACCEPT'

config rule
        option name 'Allow-ISAKMP'
        option src 'wan'
        option dest 'lan'
        option dest_port '500'
        option proto 'udp'
        option target 'ACCEPT'
4

First device is with ancient fw3 , likely from oem sdk.
What is in firewall,user file?

5

It's not quite OEM as far as I'm aware, though certainly not a proper installation. I can attempt to describe it better, but as for your request, good suggestion! Here is the output from the RM551E (the odd ancient openwrt)

root@OpenWrt:~# cat /etc/firewall.user
firewall.user           firewall.user.nat       firewall.user.ttl
firewall.user.mtu       firewall.user.tcmpmss4
root@OpenWrt:~# cat /etc/firewall.user
iptables -t filter -D FORWARD -i rmnet_data0 -p esp -j DROP
iptables -t filter -D FORWARD -i rmnet_data0 -p esp -j ACCEPT
iptables -t filter -I FORWARD -i rmnet_data0 -p esp -j ACCEPT
root@OpenWrt:~# cat /etc/firewall.user.mtu
ip link set rmnet_data0 mtu 1492
ip link set rmnet_data1 mtu 1492
ip link set rmnet_data15 mtu 1492
ip link set rmnet_data16 mtu 1492
ip link set rmnet_data2 mtu 1492
ip link set rmnet_data3 mtu 1492
ip link set rmnet_data4 mtu 1492
ip link set rmnet_data5 mtu 1492
root@OpenWrt:~# cat /etc/firewall.user.nat
root@OpenWrt:~# cat /etc/firewall.user.tcmpmss4
iptables -t mangle -A FORWARD -p tcp -o rmnet_data0 --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1432
iptables -t mangle -A FORWARD -p tcp -i rmnet_data0 --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1432
root@OpenWrt:~# cat /etc/firewall.user.ttl
root@OpenWrt:~#

The bpi-r3 doesnt have any custom firewall rules at this point, trying to keep it as close to mainline stock as possible to make it easier to diagnose.

I honestly don't know the full origins of the openwrt on this modem. Its an RM551E, which is an enginnering sample prerelease modem from Quectel (china) that may or may not be officially released at some point. I can provide this source for where I obtained the openwrt installation (it did not come with it), which can be located here:

But also worth reiterating, while I do suspect the modem/weird non-official openwrt on the modem as possibly being at least partially responsible, the modem works correctly connected directly to my desktop, so i really do feel there has to be some way to paper around an oddity on the modem by changing something on the bpi-r3 openwrt install, which i have full control over and understand the layout of much better than this bastardized unofficial openwrt on the modem (mostly stock since I just recently moved it to 24.10-RC5 and intentionally did not migrate most configs over to isolate potential cruft as a cause)

To also be fair, the old RM520 ran a different administration frontend from the same author that worked great. I honestly didnt need openwrt on the modem itself, but "simple admin" really is fantastic software and now seems to depend on that odd openwrt installation with the RM551E (apparently the RM551E has a very different layout, which makes sense).. But I still believe (without hard evidence) that some issue happening between the modem and router is occurring because of some weird obscure openwrt setting that I'm missing on the modem thats causing the router to act weird with the data coming from the modem.

When i had AI helping me, it had me do tcpdumps of the data between the modem and router and send the results, which showed a ton of ack,rst responses flooding the dump that didnt seem to make sense. That was seemingly resolved at the time, but the issue persists (and it's a fresh install again so maybe that specific problem returned? Either way, possibly a red herring but wanted to mention it)

6

Well, it is some proprietary SDK and using iptables.
The user scripts look like addressing problems long solved in fw3
Not much help on that (latency/speed characteristics of your internet link?)
You can sort away inactive rules using https://github.com/sl0/opti (probably not easy given proprietary fw will not give you packages)

If you could set up a local ethernet iperf3 server and check for any problems occurring locally.

7

I can provide a few different test results, but they can get confusing and convoluted. I'll do my best to explain.

I set up iperf3 on the modem and ran both an iperf3 test to/from the modem and an internet speed test on the bpi-r3 router.. And iperf3 is giving weird results here now.

root@OpenWrt:~# iperf3 -c 192.168.224.1 --bidir
Connecting to host 192.168.224.1, port 5201
[  5] local 192.168.225.100 port 40088 connected to 192.168.224.1 port 5201
[  7] local 192.168.225.100 port 40092 connected to 192.168.224.1 port 5201
[ ID][Role] Interval           Transfer     Bitrate         Retr  Cwnd
[  5][TX-C]   0.00-1.00   sec  13.2 MBytes   111 Mbits/sec  245   36.8 KBytes
[  7][RX-C]   0.00-1.00   sec  39.8 MBytes   333 Mbits/sec
[  5][TX-C]   1.00-2.00   sec  13.4 MBytes   112 Mbits/sec  172   31.1 KBytes
[  7][RX-C]   1.00-2.00   sec  43.6 MBytes   366 Mbits/sec
[  5][TX-C]   2.00-3.00   sec  13.1 MBytes   110 Mbits/sec  269   62.2 KBytes
[  7][RX-C]   2.00-3.00   sec  43.9 MBytes   368 Mbits/sec
[  5][TX-C]   3.00-4.00   sec  13.5 MBytes   113 Mbits/sec  287   38.2 KBytes
[  7][RX-C]   3.00-4.00   sec  42.4 MBytes   355 Mbits/sec
[  5][TX-C]   4.00-5.00   sec  13.2 MBytes   111 Mbits/sec  273   33.9 KBytes
[  7][RX-C]   4.00-5.00   sec  42.2 MBytes   354 Mbits/sec
[  5][TX-C]   5.00-6.00   sec  12.0 MBytes   101 Mbits/sec  205   31.1 KBytes
[  7][RX-C]   5.00-6.00   sec  44.1 MBytes   370 Mbits/sec
[  5][TX-C]   6.00-7.00   sec  13.5 MBytes   113 Mbits/sec  177   31.1 KBytes
[  7][RX-C]   6.00-7.00   sec  43.6 MBytes   366 Mbits/sec
[  5][TX-C]   7.00-8.00   sec  13.5 MBytes   113 Mbits/sec  202   46.7 KBytes
[  7][RX-C]   7.00-8.00   sec  43.0 MBytes   361 Mbits/sec
[  5][TX-C]   8.00-9.00   sec  13.9 MBytes   116 Mbits/sec  221   35.4 KBytes
[  7][RX-C]   8.00-9.00   sec  43.8 MBytes   367 Mbits/sec
[  5][TX-C]   9.00-10.00  sec  12.9 MBytes   108 Mbits/sec  243   48.1 KBytes
[  7][RX-C]   9.00-10.00  sec  41.6 MBytes   349 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID][Role] Interval           Transfer     Bitrate         Retr
[  5][TX-C]   0.00-10.00  sec   132 MBytes   111 Mbits/sec  2294             sender
[  5][TX-C]   0.00-10.00  sec   132 MBytes   111 Mbits/sec                  receiver
[  7][RX-C]   0.00-10.00  sec   430 MBytes   361 Mbits/sec    0             sender
[  7][RX-C]   0.00-10.00  sec   428 MBytes   359 Mbits/sec                  receiver

iperf Done.
root@OpenWrt:~# speedtest-go

    speedtest-go v1.7.7 (git-dev) @showwin

✓ ISP: 172.59.25.233 (T-Mobile USA) [41.4344, -81.6373]
✓ Found 20 Public Servers

✓ Test Server: [20130] 5.97km Cleveland, OH (United States) by T-Mobile
✓ Latency: 18.89314ms Jitter: 4.184297ms Min: 14.155292ms Max: 29.421501ms
✓ Packet Loss Analyzer: Running in background (<= 30 Secs)
✓ Download: 682.51 Mbps (Used: 831.32MB) (Latency: 61ms Jitter: 19ms Min: 24ms Max: 91ms)
✓ Upload: 14.95 Mbps (Used: 21.34MB) (Latency: 33ms Jitter: 14ms Min: 14ms Max: 53ms)
✓ Packet Loss: N/A
root@OpenWrt:~#

One of the home servers iperf3 to the modem (through the router first to the modem), iperf3 to the router, and a speedtest result.

i[blumpkin@blumpkin ~]$ iperf3 -c 192.168.224.1 --bidir
Connecting to host 192.168.224.1, port 5201
[  5] local 192.168.1.23 port 47750 connected to 192.168.224.1 port 5201
[  7] local 192.168.1.23 port 47766 connected to 192.168.224.1 port 5201
[ ID][Role] Interval           Transfer     Bitrate         Retr  Cwnd
[  5][TX-C]   0.00-1.00   sec  14.1 MBytes   118 Mbits/sec   75   29.7 KBytes
[  7][RX-C]   0.00-1.00   sec  37.6 MBytes   315 Mbits/sec
[  5][TX-C]   1.00-2.00   sec  12.8 MBytes   107 Mbits/sec   86   36.8 KBytes
[  7][RX-C]   1.00-2.00   sec  44.4 MBytes   372 Mbits/sec
[  5][TX-C]   2.00-3.00   sec  13.6 MBytes   114 Mbits/sec   71   41.0 KBytes
[  7][RX-C]   2.00-3.00   sec  43.4 MBytes   364 Mbits/sec
[  5][TX-C]   3.00-4.00   sec  14.8 MBytes   124 Mbits/sec   94   43.8 KBytes
[  7][RX-C]   3.00-4.00   sec  44.4 MBytes   372 Mbits/sec
[  5][TX-C]   4.00-5.00   sec  13.1 MBytes   110 Mbits/sec   92   33.9 KBytes
[  7][RX-C]   4.00-5.00   sec  43.9 MBytes   368 Mbits/sec
[  5][TX-C]   5.00-6.00   sec  13.8 MBytes   115 Mbits/sec   75   29.7 KBytes
[  7][RX-C]   5.00-6.00   sec  44.0 MBytes   369 Mbits/sec
[  5][TX-C]   6.00-7.00   sec  13.4 MBytes   112 Mbits/sec   61   45.2 KBytes
[  7][RX-C]   6.00-7.00   sec  44.1 MBytes   370 Mbits/sec
[  5][TX-C]   7.00-8.00   sec  12.5 MBytes   105 Mbits/sec   46   31.1 KBytes
[  7][RX-C]   7.00-8.00   sec  43.9 MBytes   368 Mbits/sec
[  5][TX-C]   8.00-9.00   sec  13.9 MBytes   116 Mbits/sec   78   49.5 KBytes
[  7][RX-C]   8.00-9.00   sec  43.6 MBytes   366 Mbits/sec
[  5][TX-C]   9.00-10.00  sec  13.6 MBytes   114 Mbits/sec   82   31.1 KBytes
[  7][RX-C]   9.00-10.00  sec  43.1 MBytes   362 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID][Role] Interval           Transfer     Bitrate         Retr
[  5][TX-C]   0.00-10.00  sec   136 MBytes   114 Mbits/sec  760             sender
[  5][TX-C]   0.00-10.00  sec   135 MBytes   113 Mbits/sec                  receiver
[  7][RX-C]   0.00-10.00  sec   434 MBytes   364 Mbits/sec    0             sender
[  7][RX-C]   0.00-10.00  sec   432 MBytes   363 Mbits/sec                  receiver

iperf Done.
[blumpkin@blumpkin ~]$ iperf3 -c 192.168.1.1 --bidir
Connecting to host 192.168.1.1, port 5201
[  5] local 192.168.1.23 port 44948 connected to 192.168.1.1 port 5201
[  7] local 192.168.1.23 port 44952 connected to 192.168.1.1 port 5201
[ ID][Role] Interval           Transfer     Bitrate         Retr  Cwnd
[  5][TX-C]   0.00-1.00   sec   210 MBytes  1.76 Gbits/sec   65   1.25 MBytes
[  7][RX-C]   0.00-1.00   sec   177 MBytes  1.48 Gbits/sec
[  5][TX-C]   1.00-2.00   sec   225 MBytes  1.89 Gbits/sec    0   1.36 MBytes
[  7][RX-C]   1.00-2.00   sec   119 MBytes   998 Mbits/sec
[  5][TX-C]   2.00-3.00   sec   227 MBytes  1.91 Gbits/sec   16   1.08 MBytes
[  7][RX-C]   2.00-3.00   sec   118 MBytes   992 Mbits/sec
[  5][TX-C]   3.00-4.00   sec   227 MBytes  1.90 Gbits/sec    0   1.22 MBytes
[  7][RX-C]   3.00-4.00   sec   117 MBytes   980 Mbits/sec
[  5][TX-C]   4.00-5.00   sec   226 MBytes  1.90 Gbits/sec    0   1.35 MBytes
[  7][RX-C]   4.00-5.00   sec   117 MBytes   985 Mbits/sec
[  5][TX-C]   5.00-6.00   sec   223 MBytes  1.87 Gbits/sec   11   1.08 MBytes
[  7][RX-C]   5.00-6.00   sec   124 MBytes  1.04 Gbits/sec
[  5][TX-C]   6.00-7.00   sec   228 MBytes  1.91 Gbits/sec    0   1.23 MBytes
[  7][RX-C]   6.00-7.00   sec   113 MBytes   944 Mbits/sec
[  5][TX-C]   7.00-8.00   sec   229 MBytes  1.92 Gbits/sec    0   1.36 MBytes
[  7][RX-C]   7.00-8.00   sec   108 MBytes   910 Mbits/sec
[  5][TX-C]   8.00-9.00   sec   216 MBytes  1.81 Gbits/sec    1   1.07 MBytes
[  7][RX-C]   8.00-9.00   sec   141 MBytes  1.18 Gbits/sec
[  5][TX-C]   9.00-10.00  sec   227 MBytes  1.91 Gbits/sec    0   1.22 MBytes
[  7][RX-C]   9.00-10.00  sec   114 MBytes   956 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID][Role] Interval           Transfer     Bitrate         Retr
[  5][TX-C]   0.00-10.00  sec  2.19 GBytes  1.88 Gbits/sec   93             sender
[  5][TX-C]   0.00-10.01  sec  2.18 GBytes  1.87 Gbits/sec                  receiver
[  7][RX-C]   0.00-10.00  sec  1.22 GBytes  1.05 Gbits/sec  1368             sender
[  7][RX-C]   0.00-10.01  sec  1.22 GBytes  1.05 Gbits/sec                  receiver

iperf Done.
[blumpkin@blumpkin ~]$ speedtest

   Speedtest by Ookla

[error] Error: [0] Timeout occurred in connect.
      Server: Cable Co-op - Oberlin, OH (id: 25491)
         ISP: T-Mobile USA
Idle Latency:    35.00 ms   (jitter: 0.43ms, low: 34.60ms, high: 35.48ms)
    Download:   893.11 Mbps (data used: 1.5 GB)                                           
                184.24 ms   (jitter: 54.12ms, low: 33.28ms, high: 632.67ms)
      Upload:     7.85 Mbps (data used: 11.9 MB)                                          
                 38.53 ms   (jitter: 8.93ms, low: 31.77ms, high: 367.09ms)
 Packet Loss:     0.0%
  Result URL: https://www.speedtest.net/result/c/32f9585d-794b-4ce3-af4f-074140f518f1
[blumpkin@blumpkin ~]$

If iperf3 to the modem from the router is actually only giving 350mbps, how is it giving almost a gigabit during the speed test?

Speedtest on the modem:

root@OpenWrt:~# speedtest

   Speedtest by Ookla

      Server: T-Mobile - Cleveland, OH (id: 20130)
         ISP: T-Mobile USA
Idle Latency:    19.28 ms   (jitter: 4.04ms, low: 14.99ms, high: 24.36ms)
    Download:  1901.03 Mbps (data used: 2.0 GB)                                           
                179.59 ms   (jitter: 57.46ms, low: 20.99ms, high: 576.40ms)
      Upload:   162.22 Mbps (data used: 283.6 MB)                                         
                 40.39 ms   (jitter: 7.95ms, low: 13.12ms, high: 147.99ms)
 Packet Loss:     0.0%
  Result URL: https://www.speedtest.net/result/c/be3b093c-6a7a-4bff-9b3a-bdc16e40dcf8
root@OpenWrt:~#

These numbers make no sense to me, if the link between the modem and router is only 350mbps, how are client devices getting a gigabit?

Oh! Important note that I neglected to mention, my ISP being tmobile means it's pure IPV6, and the modem also handles CLAT/XLAT64.. Resources are not notably high when running full throttle using ip passthrough (but high when not, as there doesnt seem to be any offload in that scenario).

Please, by all means, if you need more clarification or have any additional ideas of things to try or info to present, I am absolutely at your mercy. And thank you (seriously!) for your guidance so far.

8

speedtest directly wired to "modem" using normal PC, not a router as client.

9

Thats a little trickier to perform at the moment, as the bpi-r3 is behind the fridge up on the ceiling and needs a stepladder to access, and the poe line to the modem is right there. I'll get these results for you later tonight (my apologies)

10

It is to test at random times over the day, so far nothing points to openwrt faults.

11

Fault might not be the right word, i certainly don't think it's a "fault" of one device or another, to be completely fair.. My suspicion is theres a configuration issue with the modems weird openwrt thats conflicting with the routers mainline openwrt in an odd way, and since im very nervous about messing with the modem, Im reluctant to play with it too much for fear of breaking it and having to climb an icy snowy ladder to pull the modem to fix it.. So most of my focus has been on making the router "adapt" to whatever oddity the modem pushes out.

Also another reason I have to wait until later is the jellyfin server serves about 40 people, and I try to keep disruptive diagnostics limited to later at night to minimize complaints, among household approval factor aspects (housemates complain about poor network conditions but are extremely intolerant of visible diagnostic work, so pulling the fridge out requires waiting for other household members to go to bed.. Fun times)

12

Likely slowdowns are related to switching bands or cell towers, probably there is some log on that router-modem. There is nothing intentionally clamping bw on openwrt 10x at random.

13

Its almost certainly not the actual cell network from what Ive gathered. I'm under 500m from my local tower, and the next nearest tower is roughly 25km away. I have 210mhz of NR bandwidth, and while occasionally the band locks get mucky (and always have), these tests were all run immediately after resetting the band locks.

I can reliably pull roughly 2gbps down at any given time on the modem itself while, at the same time, not be able to clear 100mbps on other devices on the network. Its very rural, this tower has few users and the tests on the modem itself pretty consistently confirm that the link from the modem to the tower is quite solid.

Simultaneous speed tests on multiple devices are consistent in their behavior. The previous modem didnt have these issues and regularly could do 1gbps on every device (which was roughly what that modem was capable of from tower to modem)

These issues started with the "upgrade" to the RM551E.

And i dont believe its any kind of intentional throttle anywhere, because inter-device consistency would make that easier to locate.

I cant connect to PIA on this phone on wifi (fine on cellular), but can on my other phone on the same wifi. The whole network is suddenly rife with inconsistent behavior that i just can't nail down a source of since installing the RM551e as the new modem. Maybe i should cut my losses and reinstall the RM520 and just sell this thing. The extra speed is nice, but clearly i'm in way over my head with this modem.. It's junking up something with the traffic, but figuring out specifically what has proven to be a monumental challenge.