Unrelated, but If your adapter has a USB type C to USB 3 try inverting the USB C connector I seen performance increase doing this when I connect an external ssd drive.
1 Like
Found it,
From the reading i did last night with the new interfaces 2.5GB + you can't force the speed and duplex. Lets face it unless there was a bug somewhere forcing the speeds just caused network hangs any way.
The fix is to control the advertising
I added
ethtool -s eth1 autoneg on advertise 0x80000000002f
to /etc/rc.local on the openwrt snapshot 21.02.3
and it works.
Speed: 2500Mb/s
Duplex: Full
Auto-negotiation: on
My goal here is to be able to route almost 1.2gps, with a 2.5GB lan interface, then bonding two 1GB nics (the built in one and another USB one.)
If anyone is interested I will post the results.
3 Likes
Please do as I will be doing similar possibly
So far i built a pi4 B (8GB) in three configurations
Config 1
Openwrt 21.02
Current OpenWRT released build 21.02
1 gig usb dongle(wan) and network port(lan).
Enabled software acceleration / irq balancing and receive packet steering
Modules installed: kmod-boding , kmod-usb-net-ax88179 , mii-tool , mwan3
On a near 1.2Gb connection got about 840-860 mbs
Very low cpu utilisation, sirq in the low percent , can’t see where the bottle next is, but I must be getting close the net port max especially when you take in to account overheads.
I read on the forums that if you bond the wan connections, you may get more.
Config 2
Openwrt 21.02
2 x 1 gig usb dongle(wan) bonded round robin and network port(lan).
Enabled software acceleration / irq balancing and receive packet steering / mwan3
Modules installed kmod-boding , kmod-usb-net-ax88179 , mii-tool , mwan3
On a near 1.2Gb connection got about 890-920 mbs
That’s better, I could see the traffic going across both wan ports (not evenly balanced but it works)
This now points the lan port
Config 3
Openwrt 21.03
1 x 1 gig usb dongle(wan) + onboard network bonded round robin
1x2.5gb usb dongle for lan
Enabled software acceleration / irq balancing and receive packet steering
Modules installed kmod-boding , kmod-usb-net-ax88179 , mii-tool , mwan3, r8152 drivers
On a near 1.2Gb connection got about 290 - 320 mbs (that’s the magic number for USB2 , but the dongles are using the usb3 ports) I know they both share a single channel , but a single channel on USB3 is big enough)
I did have a lot of issues, but this is beta s/w.
The issues so far:
Openwrt 21.02 had an old driver for r8252. The new r8152 driver also support the R8256b (my chip in the 2.5gb dongle).
The 2.5gb interface by default runs at 1000/full and required the advertised links to be changed (see above)
Mwan3 complains about iptables-legacy and iptables-legacy6 - Resolved by installing the modules
The usb nic for the bonding at boot stays down, and hence the bond will not start, When you do start it the shell freezes and disconnects, even though I am using the wan. most strange
Requires ip link set ethx up
The poor performance (still looking into this)
1 Like
Think i'm getting there. I'm trying to set up a round robin bond for virgin
using this as a guide
if i turn arp off on the bond using
ip link set ${interface name} arp off
nothing gets transferred (fair enough). How do i set up manual arp entries for bond 0 which uses eth0 and eth2. I believe that are looking for a single arp for the bond. Currently the arp for the bond is the same as eth0
Think i am going to to give up on this for a while. It appears no one knows the answer above. With iperf3 i can get 2.3 Gb down and about 1.5gb up, but i know there are issues with the driver. Alsop when running in a bond i'm loosing 50% throughput, disconnecting either cable instantly increased the throughput by 100%
You are doing better than I have been. If I lock the adapter in my laptop to 1Gbps I get that speed from iperf3. If I let it autonegotiate to 2.5 best I can get from iperf is 608Mbps...
That is on my own compile though.
Linux ghost 5.15.45 #0 SMP Sat Jun 11 19:22:58 2022 x86_64 GNU/Linux
What's been pointed out before, there's a reason why you don't see Realtek or USB at all in network appliances where stability and throughput is a priority. There are also reports of 2.5G adapters running hot leading to various interesting issues. What you could try is to use CDC-NCM mode (which Realtek controllers "should" support) to see if that helps in terms of performance but I wouldn't hold my breath. According to https://en.wikipedia.org/wiki/USB_3.0 effective throughput is about 4Gbit/s on USB 3.0 although I think it's probably safe to say that your run of the mill cheap as chips SoC wont do that reliably so what you're getting running iperf3 is most likely about the limit of what the hardware can do.
1 Like
Mine is currently using cdc-ncm which at least on my custom build is broken somewhat it seems because ethtool is not happy.
root@ghost:/mnt/usb1/openwrt/bin/packages/x86_64/packages# ethtool eth3
Settings for eth3:
Supported ports: [ ]
Supported link modes: Not reported
Supported pause frame use: No
Supports auto-negotiation: No
Supported FEC modes: Not reported
Advertised link modes: Not reported
Advertised pause frame use: No
Advertised auto-negotiation: No
Advertised FEC modes: Not reported
Speed: 2500Mb/s
Duplex: Half
Auto-negotiation: off
Port: Twisted Pair
PHYAD: 0
Transceiver: internal
MDI-X: Unknown
Current message level: 0x00000007 (7)
drv probe link
Link detected: yes
root@ghost:/mnt/usb1/openwrt/bin/packages/x86_64/packages# ip link show dev eth3
15: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br-lan state UP mode DEFAULT group default qlen 1000
link/ether 9c:b6:d0:3c:20:6e brd ff:ff:ff:ff:ff:ff
root@ghost:/mnt/usb1/openwrt/bin/packages/x86_64/packages# lsusb
Bus 001 Device 002: ID 0438:7900
Bus 003 Device 008: ID 152d:1561 SABRENT SABRENT
Bus 001 Device 001: ID 1d6b:0002 Linux 5.15.45 ehci_hcd EHCI Host Controller
Bus 003 Device 007: ID 1a56:3100 Rivet USB 10/100/1G/2.5G LAN
Bus 003 Device 006: ID 2109:8110 VIA Labs, Inc. USB3.0 Hub
Bus 002 Device 003: ID 2109:2811 VIA Labs, Inc. USB2.0 Hub
Bus 002 Device 001: ID 1d6b:0002 Linux 5.15.45 xhci-hcd xHCI Host Controller
Bus 003 Device 001: ID 1d6b:0003 Linux 5.15.45 xhci-hcd xHCI Host Controller
root@ghost:/mnt/usb1/openwrt/bin/packages/x86_64/packages# lsusb -t
/: Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/2p, 5000M
|__ Port 1: Dev 6, If 0, Class=, Driver=hub/4p, 5000M
|__ Port 2: Dev 8, If 0, Class=, Driver=usb-storage, 5000M
|__ Port 4: Dev 7, If 0, Class=, Driver=cdc_ncm, 5000M
|__ Port 4: Dev 7, If 1, Class=, Driver=cdc_ncm, 5000M
/: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/2p, 480M
|__ Port 1: Dev 3, If 0, Class=, Driver=hub/4p, 480M
/: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/2p, 480M
|__ Port 1: Dev 2, If 0, Class=, Driver=hub/4p, 480M
What I have found today incontrovertably is @slh is correct. APU2 is on verge of EOL. iperf3 or iperf running 1 instance cpu limits the speed when a 2.5Gbps adapter gets used at a link speed of 2.5Gbps.
If I use the router as iperf 2 or 3 client I can get 1.1Gbps over the 2.5Gbps link. So looks like its time to upgrade.
As for my adapters;
Killer E3100 USB-C 3.1 to RJ-45 2.5Gbps Ethernet Adapter (100123) Realtek R8169 based
ASUS 2.5G Ethernet USB Adapter (USB-C2500) Realtek R8152 based
Both have issues on other linux platforms as well. I can barely get them functional on manjaro using just normal repo modules/drivers/firmware
My only other possible way to get more than 1Gbps out of this is something like https://www.aliexpress.com/item/3256803346704168.html?gatewayAdapt=4itemAdapt
Its a damn shame, the APU is a great unit, but with my inbound traffic capable of 1.2Gbps I want more capability.
I have a usb3 2.5gbit Realtek 8125, it's capable of delivering the speed advertised, w/o any hiccups.
Not using openwrt on the machines (x86s) where I made the tests though, but it could be arranged.
I would imagine that cdc-ncm cuts some corners in terms of functionality and that you would need to use the chipset driver instead. iperf3 most likely uses too much resources on one core to accurately test NIC performance at those speeds on such "slow" hardware.
So, looks like the speed issue is the CPU and crappy scaling (apparently way too sluggish) because when I disable the scaling. I can get good speeds. 2.5Gbps using rsync over an nfs share to a windows machine.So I am happy now. I dont care if I disable scaling because this thing is cooled well, and does great.
root@ghost:~/testfile# sensors
mt7915_phy1-pci-0400
Adapter: PCI adapter
temp1: +70.0°C (high = +120.0°C, crit = +110.0°C)
k10temp-pci-00c3
Adapter: PCI adapter
temp1: +49.0°C (high = +70.0°C)
(crit = +105.0°C, hyst = +104.0°C)
acpitz-acpi-0
Adapter: ACPI interface
temp1: +49.0°C (crit = +115.0°C)
mt7915_phy0-pci-0400
Adapter: PCI adapter
temp1: +70.0°C (high = +120.0°C, crit = +110.0°C)
fam15h_power-pci-00c4
Adapter: PCI adapter
power1: 1.80 W (interval = 0.01 s, crit = 6.00 W)
Yeah I have no doubt the cdc-ncm cuts some corners, would be nice to be able to see any info ethtool.
Hello
I still have problem by spd480, any ideas ??
T: Bus=05 Lev=02 Prnt=02 Port=02 Cnt=01 Dev#= 3 Spd=480 MxCh= 0
D: Ver= 2.10 Cls=00(>ifc ) Sub=00 Prot=00 MxPS=64 #Cfgs= 3
P: Vendor=0bda ProdID=8156 Rev=31.04
S: Manufacturer=Realtek
S: Product=USB 10/100/1G/2.5G LAN
S: SerialNumber=4013000001
C:* #Ifs= 1 Cfg#= 1 Atr=a0 MxPwr=200mA
I:* If#= 0 Alt= 0 #EPs= 3 Cls=ff(vend.) Sub=ff Prot=00 Driver=r8152
E: Ad=81(I) Atr=02(Bulk) MxPS= 512 Ivl=0ms
E: Ad=02(O) Atr=02(Bulk) MxPS= 512 Ivl=0ms
E: Ad=83(I) Atr=03(Int.) MxPS= 2 Ivl=128ms
C: #Ifs= 2 Cfg#= 2 Atr=a0 MxPwr=200mA
I: If#= 0 Alt= 0 #EPs= 1 Cls=02(comm.) Sub=0d Prot=00 Driver=
E: Ad=83(I) Atr=03(Int.) MxPS= 16 Ivl=128ms
I: If#= 1 Alt= 0 #EPs= 0 Cls=0a(data ) Sub=00 Prot=01 Driver=
I: If#= 1 Alt= 1 #EPs= 2 Cls=0a(data ) Sub=00 Prot=01 Driver=
E: Ad=81(I) Atr=02(Bulk) MxPS= 512 Ivl=0ms
E: Ad=02(O) Atr=02(Bulk) MxPS= 512 Ivl=0ms
C: #Ifs= 2 Cfg#= 3 Atr=a0 MxPwr=200mA
I: If#= 0 Alt= 0 #EPs= 1 Cls=02(comm.) Sub=06 Prot=00 Driver=
E: Ad=83(I) Atr=03(Int.) MxPS= 16 Ivl=128ms
I: If#= 1 Alt= 0 #EPs= 0 Cls=0a(data ) Sub=00 Prot=00 Driver=
I: If#= 1 Alt= 1 #EPs= 2 Cls=0a(data ) Sub=00 Prot=00 Driver=
E: Ad=81(I) Atr=02(Bulk) MxPS= 512 Ivl=0ms
E: Ad=02(O) Atr=02(Bulk) MxPS= 512 Ivl=0msUsing the wrong USB port?
No no, i checked all
Y\This card working ok in Windows
This same port i put other 1 Gbit card and spd=5000
Only i OpenWRT ( Windows also ) i use cable USB C to USB 3.0
I have card USB C
Try turning the USB-C connector 180°. In my experience, these A-to-C adapters are often not symmtrical. They only have USB 3 signals on one side of the USB-C connector
2 Likes
Very thank
oh my good, 180 and working ok 
I think yet have to small power in usb 3,0 - i try download and in moment i have restart USB 
But 2500 working ok
```
root@OpenWrt:~# ethtool -s eth2 autoneg on advertise 0x80000000002f
root@OpenWrt:~# ethtool eth2
Settings for eth2:
Supported ports: [ MII ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
2500baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
2500baseT/Full
Advertised pause frame use: Symmetric Receive-only
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Link partner advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
2500baseT/Full
Link partner advertised pause frame use: No
Link partner advertised auto-negotiation: Yes
Link partner advertised FEC modes: Not reported
Speed: 2500Mb/s
Duplex: Full
Port: MII
PHYAD: 32
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: pumbg
Wake-on: g
Current message level: 0x00007fff (32767)
drv probe link timer ifdown ifup rx_err tx_err tx_queued intr tx_done rx_status pktdata hw wol
Link detected: yesWow, that wouldn't have crossed my mind :o
Edit
works ok - not enabled switch to other side
Hello, i have new problem
If uses driver 8152, and start download i have in 1-2 sec error and broken session
if i use driver cdc_ncm working ok, but not possible 2500 GBits for this, any ideas ?
[ 480.649669] r8152 6-2.1:1.0 eth1: carrier on
[ 486.288784] pppoe-wan: renamed from ppp0
[ 570.382771] xhci_hcd 0000:00:10.0: WARN: TRB error for slot 3 ep 3 on endpoint
[ 570.390069] xhci_hcd 0000:00:10.0: WARN waiting for error on ep to be cleared
[ 570.390074] r8152 6-2.1:1.0 eth1: Tx status -84
[ 570.401863] r8152 6-2.1:1.0 eth1: failed tx_urb -22
[ 570.407040] xhci_hcd 0000:00:10.0: WARN waiting for error on ep to be cleared
[ 570.414252] r8152 6-2.1:1.0 eth1: failed tx_urb -22
[ 570.419297] xhci_hcd 0000:00:10.0: WARN waiting for error on ep to be cleared
and more more....
If i use cdc_ncm but working ok but only link ( 1 GBits )
[ 12.420539] cdc_ncm 6-2.1:2.0 eth1: 1000 mbit/s downlink 1000 mbit/s uplink
[ 12.484919] IPv6: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
[ 12.493555] IPv6: ADDRCONF(NETDEV_CHANGE): eth1.35: link becomes ready
[ 13.457485] pppoe-wan: renamed from ppp0