High Latency Packets

I have a Raspberry Pi 3B setup with OpenWRT as a network bridge for a USB LTE Modem. About 1 in 15 packets takes multiple seconds to respond and it has been narrowed to the connection between the router and Raspberry Pi. I have replaced the cable and the Modem Manager process takes a consistent 2% CPU time in top plus only about 200Mb/1000Mb are used from the memory so I have not been able to find a reason for the random high Latency Packets but it makes it nearly completely unusable and I am hoping there is a solution to resolve this without spending another $200 on a Raspberry Pi 5 setup.

What is the "router" here? The providers one at the base of gsm tower?

Not Internet Providers, it's TPLink AXE5400. Unfortunately the better ones cost $1000+ which is out of budget with a significant amount already spent trying to fix the internet distribution to my network, I can't get any fixed internet service so am relying on LTE service by a USB modem which the Rasberry Pi needs to act as a bridge for.

Raspberry with one ethernet port can not serve as any bridge, it can work as mini server of all trades.
How do you measure latency?

Latency is measured with Ping to Rasberry Pi, as well as Rasberry Pi out to 8.8.8.8 via ssh session which included delays with the ssh packets updating but this delays are not represented in the pings sent from the Rasberry Pi itself.

Try this:
https://www.waveform.com/tools/bufferbloat

1 Like

The 3B uses USB2.0 attached fastEthernet, so is limited to 100 Mbps, the 3B+ uses USB2.0 attached GigabitEthernet and tops out at 300 Mbps. Which version it is? (Neither is ideal for your purpose depending on the speed over the LTE link).

As far as I can tell this is not supported by OpenWrt... that means you likely can not debug the issue from both sides.

Could you install ethtool (opkg update ; opkg install ethtool-full) and post the output of the following commands (replace eth0 with the true interface name):

ethtool eth0
ethtool --statistics eth0
ethtool --show-eee eth0

maybe this implies some ethernet related issues...

1 Like

cat /proc/net/softnet_stat too

root@OpenWrt:/dev# ethtool eth0
Settings for eth0:
	Supported ports: [ TP	 MII ]
	Supported link modes:   10baseT/Half 10baseT/Full
	                        100baseT/Half 100baseT/Full
	Supported pause frame use: Symmetric Receive-only
	Supports auto-negotiation: Yes
	Supported FEC modes: Not reported
	Advertised link modes:  10baseT/Half 10baseT/Full
	                        100baseT/Half 100baseT/Full
	Advertised pause frame use: Symmetric Receive-only
	Advertised auto-negotiation: Yes
	Advertised FEC modes: Not reported
	Link partner advertised link modes:  10baseT/Full
	                                     100baseT/Full
	Link partner advertised pause frame use: Symmetric Receive-only
	Link partner advertised auto-negotiation: Yes
	Link partner advertised FEC modes: Not reported
	Speed: 100Mb/s
	Duplex: Full
	Auto-negotiation: on
	Port: Twisted Pair
	PHYAD: 1
	Transceiver: internal
	MDI-X: Unknown
	Supports Wake-on: pumbag
	Wake-on: d
        Current message level: 0x00000007 (7)
                               drv probe link
	Link detected: yes

root@OpenWrt:/dev# ethtool --statistics eth0
no stats available

root@OpenWrt:/dev# ethtool --show-eee eth0
netlink error: Not supported

root@OpenWrt:/dev# cat /proc/net/softnet_stat
0000092a 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 000002fb 00000000 00000000 00000000
00000263 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 0000006d 00000000 00000000 00000001
00000354 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 0000013f 00000000 00000000 00000002
000002ef 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 000000e2 00000000 00000000 00000003

root@OpenWrt:/dev# uqmi -d /dev/cdc-wdm0 --get-signal-info
^C"Failed to connect to service"

Unfortunately there is a new issue, uqmi has randomly broken despite nothing changing. It hangs and when aborted returns either "error" or "Failed to connect" but occasionally will work for 1 or 2 commands after a usbreset

You reboot too often to diagnose anything. Few thousand packets processed is not indicative of anything.

I would for example investigate in the direction of packet offloading. If it's the bandwidth, than take a look at cake-autorate. The latter had a big impact in my docsis link!