[Solved] R8000+RTL8156: USB fails when WAN port unpopulated

[pipe2null]

Swapping to a different/1G usb dongle does not appear to exhibit this weirdness. But then it's only 1G instead of 2.5G, which defeats the point.

So... This is weird. When there is SOMEthing, ANYthing plugged into the WAN port, the USB dongle works fine-ish. By ANYthing, even a plain network cable not connected to anything else, as in the 2nd RJ45 flopping around in the open air, this works fine. But the moment I UN-plug anything from the WAN port, the USB dongle starts failing, appears to lose connectivity, LEDs go off, etc.

I'm not 100% sure, but it seems that connecting something to the WAN port that is also connected to a different switch (the other switch not connected to anything else) has better USB stability than when an unconnected cable is plugged into WAN, and in either case the USB appears to work ok but eventually runs into connectivity issues. If nothing is plugged into WAN, the device will boot up, start the USB dongle, work ok for a few seconds, and then crap out the same way it does if the WAN is unplugged later after bootup completes.

Setup:
Netgear R8000
USB 3.0 port-> RTL8156 usb to 2.5G dongle

Model	Netgear R8000 (BCM4709)
Architecture	ARMv7 Processor rev 0 (v7l)
Target Platform	bcm53xx/generic
Firmware Version	OpenWrt 22.03.2 r19803-9a599fee93 / LuCI openwrt-22.03 branch git-22.288.45147-96ec0cd
Kernel Version	5.10.146

:~# lsusb
Bus 004 Device 002: ID 0bda:8156 Realtek USB 10/100/1G/2.5G LAN

dmesg spew from unplugging WAN port (NOT the usb dongle) and replugging WAN in:

[ 2194.177741] usb 4-1: USB disconnect, device number 7
[ 2194.182847] xhci-hcd 18023000.usb: WARN Set TR Deq Ptr cmd failed due to incorrect slot or ep state.
[ 2194.192048] cdc_ncm 4-1:2.0 eth3: unregister 'cdc_ncm' usb-18023000.usb-1, CDC NCM
[ 2194.199981] xhci-hcd 18023000.usb: WARN Set TR Deq Ptr cmd failed due to incorrect slot or ep state.
[ 2194.618062] usb 4-1: new SuperSpeed Gen 1 USB device number 8 using xhci-hcd
[ 2194.668125] cdc_ncm 4-1:2.0: MAC-Address: 1c:bf:ce:fb:e1:ff
[ 2194.673726] cdc_ncm 4-1:2.0: setting rx_max = 16384
[ 2194.678903] cdc_ncm 4-1:2.0: setting tx_max = 16384
[ 2194.684578] cdc_ncm 4-1:2.0 eth3: register 'cdc_ncm' at usb-18023000.usb-1, CDC NCM, 1c:bf:ce:fb:e1:ff
[ 2195.197849] usb 4-1: USB disconnect, device number 8
[ 2195.203049] xhci-hcd 18023000.usb: WARN Set TR Deq Ptr cmd failed due to incorrect slot or ep state.
[ 2195.212349] cdc_ncm 4-1:2.0 eth3: unregister 'cdc_ncm' usb-18023000.usb-1, CDC NCM
[ 2195.627936] usb 4-1: new SuperSpeed Gen 1 USB device number 9 using xhci-hcd
[ 2195.678316] cdc_ncm 4-1:2.0: MAC-Address: 1c:bf:ce:fb:e1:ff
[ 2195.683945] cdc_ncm 4-1:2.0: setting rx_max = 16384
[ 2195.692334] cdc_ncm 4-1:2.0: setting tx_max = 16384
[ 2195.704294] cdc_ncm 4-1:2.0 eth3: register 'cdc_ncm' at usb-18023000.usb-1, CDC NCM, 1c:bf:ce:fb:e1:ff
[ 2197.267889] usb 4-1: USB disconnect, device number 9
[ 2197.273009] xhci-hcd 18023000.usb: WARN Set TR Deq Ptr cmd failed due to incorrect slot or ep state.
[ 2197.282386] cdc_ncm 4-1:2.0 eth3: unregister 'cdc_ncm' usb-18023000.usb-1, CDC NCM
[ 2199.627919] usb 4-1: new SuperSpeed Gen 1 USB device number 10 using xhci-hcd
[ 2199.678165] cdc_ncm 4-1:2.0: MAC-Address: 1c:bf:ce:fb:e1:ff
[ 2199.683765] cdc_ncm 4-1:2.0: setting rx_max = 16384
[ 2199.688935] cdc_ncm 4-1:2.0: setting tx_max = 16384
[ 2199.694556] cdc_ncm 4-1:2.0 eth3: register 'cdc_ncm' at usb-18023000.usb-1, CDC NCM, 1c:bf:ce:fb:e1:ff
[ 2200.207939] usb 4-1: USB disconnect, device number 10
[ 2200.213145] xhci-hcd 18023000.usb: WARN Set TR Deq Ptr cmd failed due to incorrect slot or ep state.
[ 2200.222556] cdc_ncm 4-1:2.0 eth3: unregister 'cdc_ncm' usb-18023000.usb-1, CDC NCM
[ 2200.648135] usb 4-1: new SuperSpeed Gen 1 USB device number 11 using xhci-hcd
[ 2200.698569] cdc_ncm 4-1:2.0: MAC-Address: 1c:bf:ce:fb:e1:ff
[ 2200.704191] cdc_ncm 4-1:2.0: setting rx_max = 16384
[ 2200.709454] cdc_ncm 4-1:2.0: setting tx_max = 16384
[ 2200.715214] cdc_ncm 4-1:2.0 eth3: register 'cdc_ncm' at usb-18023000.usb-1, CDC NCM, 1c:bf:ce:fb:e1:ff
[ 2202.028005] usb 4-1: USB disconnect, device number 11
[ 2202.033219] cdc_ncm 4-1:2.0 eth3: unregister 'cdc_ncm' usb-18023000.usb-1, CDC NCM
[ 2203.438004] usb 4-1: new SuperSpeed Gen 1 USB device number 12 using xhci-hcd
[ 2203.488257] cdc_ncm 4-1:2.0: MAC-Address: 1c:bf:ce:fb:e1:ff
[ 2203.493854] cdc_ncm 4-1:2.0: setting rx_max = 16384
[ 2203.499042] cdc_ncm 4-1:2.0: setting tx_max = 16384
[ 2203.504647] cdc_ncm 4-1:2.0 eth3: register 'cdc_ncm' at usb-18023000.usb-1, CDC NCM, 1c:bf:ce:fb:e1:ff
[ 2206.299492] cdc_ncm 4-1:2.0 eth3: 2500 mbit/s downlink 2500 mbit/s uplink
[ 2206.557546] IPv6: ADDRCONF(NETDEV_CHANGE): eth3.1: link becomes ready
[ 2206.564273] IPv6: ADDRCONF(NETDEV_CHANGE): eth3.666: link becomes ready
[ 2206.571318] IPv6: ADDRCONF(NETDEV_CHANGE): eth3.911: link becomes ready

I'm transitioning this router into an AP plus local switch for printers and whatnot that may or may not be powered all the time, and using the 2.5G usb dongle as the trunk. Basically, guaranteeing something will be connected and powered on the WAN port (to be bridged with LAN ports) isn't something I was intending.

Any ideas on how to fix this without requiring something in the WAN port, and still getting my 2.5G's worth of trunk?

[andyboeh]

It doesn't really match your symptoms, but could it be a power issue (i.e. USB power, not the router's supply)? Do you have a powered USB hub at hand that you could try?

EDIT: Did I get that right that you have actually two USB devices plugged in?

1. RTL8156
2. WWAN

But when you mention WAN first, you mean the wired WAN port and when you talk about WAN later, you mean the USB WWAN card, is that correct?

EDIT2: Did you do any speed measurements? Can this device really benefit from 2.5G via USB with its limited processing power? Wouldn't it make more sense to configure the WAN port as a LAN port and use that as a trunk?

[pipe2null]

The "WAN" port is the standard built-in RJ45 port like every router on the planet, plugged straight in to the PCB, NOT a usb connected adapter. That is why it is so weird, the normal switch's WAN port should have no interaction with the USB dongle.

... Unless there is a bug in the relatively new-ish RTL8156 2.5G driver, or there is something unusual about the R8000 platform... Or like you mentioned, perhaps might be a USB power consumption issue that for whatever reason is affected by something being plugged into the WAN port?!? So, it could be a power problem, I'm thinking the higher BW ethernet burns more juice, or it is driver related and/or could be combination of installed packages, or ... Yup, it's weird.

I've been running the exact same setup for ~12hours with a different USB 1G dongle, same cables and router configuration, but without the WAN port plugged in (I have ISP connection on a vlan feeding in over the USB connected trunk). It has been working perfectly for hours now without the 2.5G dongle/WAN port interaction problem. I ordered another/different brand 2.5G dongle I'll try out as soon as it arrives. Will also try with a usb HDD to see it it's obviously a power issue.

Perf speed wise, haven't gotten to the point of running iperf, so no idea what throughput to expect with a 2.5G adapter on this router. But for $20-30 for a 2.5G dongle, if I get at least 1.3Gbps I'll be ahead. I am assuming that speed will increase a bit more when I eventually convert the unit to AP without any firewall.

Thanks for the suggestions, I'm working on them tomorrow.

[pipe2null]

So, appears to be a sub-par dongle?

The questionable 2.5G dongle has a brand "Fenvi", and appears to work as expected when connected to my laptop, but starts getting weird when used on the router when the router's WAN port is disconnected. I swapped for a newly arrived 2.5G dongle from "Plugable", and the plugable works just fine, zero interaction with WAN port connects/disconnects, as it should be. Both dongles are RTL8156, and the usb descriptors are identical other than serial number and MAC.

Conclusion: Not all RTL8156 are created equal, which is obvious, but never would expect the weird WAN port interaction. I wonder if the funky dongle is hitting an edge case/bug in the driver? For $20 for a different dongle, not really worth digging into deeper.

Perf: @andyboeh you are correct to scratch your head on actual, usable throughput. With some quick iperf tests the 2.5G usb dongle is at best on par with a 1G LAN port, with plenty of room for degraded perf, at least when testing purely wired network load. My quick tests are not an accurate representation of the usable bandwidth, but enough to indicate significant effort is not merited for this hardware.

Thanks for the help. If nothing else, I think it's freakin cool that you can upgrade your router hardware with a USB dongle, a scenario I'm pretty sure is not supported with normal OEM firmware. Yet another reason why openwrt is awesome.