Support for SFP module OEM SFP-2.5G(-T-R-RM)

Betonmischer · December 17, 2024, 12:45pm

OEM SFP-2.5G and its variant OEM SFP-2.5G-T-R-RM are 2.5GbaseT SFP modules based on the RTL8221B-VB-CG phy.

Abridged ethtool output:

root@router:~# ethtool -m eth2
	Identifier                                : 0x03 (SFP)
	Extended identifier                       : 0x04 (GBIC/SFP defined by 2-wire interface ID)
	Connector                                 : 0x07 (LC)
	Transceiver codes                         : 0x00 0x01 0x00 0x00 0x00 0x00 0x02 0x00 0x00
	Transceiver type                          : SONET: OC-48, short reach
	Encoding                                  : 0x05 (SONET Scrambled)
	BR, Nominal                               : 2500MBd
	Rate identifier                           : 0x00 (unspecified)
	Length (SMF,km)                           : 0km
	Length (SMF)                              : 0m
	Length (50um)                             : 300m
	Length (62.5um)                           : 200m
	Length (Copper)                           : 0m
	Length (OM3)                              : 0m
	Laser wavelength                          : 850nm
	Vendor name                               : OEM
	Vendor OUI                                : 00:00:00
	Vendor PN                                 : SFP-2.5G-T-R-RM
	Vendor rev                                : 1.0
	Option values                             : 0x00 0x1a
	Option                                    : RX_LOS implemented
	Option                                    : TX_FAULT implemented
	Option                                    : TX_DISABLE implemented

The one I have works out of the box on the latest OpenWrt snapshot in a Banana Pi BPI-R4. But It would occasionally lose connectivity after reboot with no error reported until it is reseated in the SFP cage or reset repeatedly with ip link set eth1 down && ip link set eth1 up.

There is a driver patch targeted at this problem, which hasn't been merged for a while: https://patchwork.kernel.org/project/netdevbpf/patch/20240303102848.164108-7-ericwouds@gmail.com/. Yet, after I inserted the fixup lines into sfp.c, the module stopped working entirely with an sfp_add_phy failed: -EOPNOTSUPP error.

[ 1263.978037] mtk_soc_eth 15100000.ethernet eth2: configuring for inband/sgmii link mode
[ 1263.985986] mtk_soc_eth 15100000.ethernet eth2: major config sgmii
[ 1263.992168] mtk_soc_eth 15100000.ethernet eth2: phylink_mac_config: mode=inband/sgmii/none adv=00,00000000,00000000,000060ef pause=01
[ 1264.007331] sfp sfp1: SM: enter present:down:down event dev_up
[ 1264.013165] sfp sfp1: tx disable 1 -> 0
[ 1264.017019] sfp sfp1: SM: exit present:up:wait
[ 1267.731616] sfp sfp1: los 1 -> 0
[ 1267.734861] sfp sfp1: SM: enter present:up:wait event los_low
[ 1267.740605] sfp sfp1: SM: exit present:up:wait
[ 1289.183300] sfp sfp1: SM: enter present:up:wait event timeout
[ 1289.189871] mdio_bus i2c:sfp1: probed
[ 1289.896408] mtk_soc_eth 15100000.ethernet eth2: interface 2 (mii) rate match none supports 0-3,6-7,13-14
[ 1289.906040] mtk_soc_eth 15100000.ethernet eth2: interface 3 (gmii) rate match none supports 0-3,5-7,13-14
[ 1289.915692] mtk_soc_eth 15100000.ethernet eth2: interface 4 (sgmii) rate match none supports 0-3,5-7,13-14
[ 1289.925428] mtk_soc_eth 15100000.ethernet eth2: interface 22 (1000base-x) rate match none supports 5-7,13-14
[ 1289.935331] mtk_soc_eth 15100000.ethernet eth2: interface 23 (2500base-x) rate match none supports 6-7,13-14,47
[ 1289.945538] mtk_soc_eth 15100000.ethernet eth2: interface 29 (usxgmii) rate match none supports 0-3,5-7,13-14,47
[ 1289.955798] mtk_soc_eth 15100000.ethernet eth2: requesting link mode inband/sgmii with support 00,00000000,00008000,000060ef
[ 1290.043974] sfp sfp1: sfp_add_phy failed: -EOPNOTSUPP
[ 1290.049036] sfp sfp1: SM: exit present:up:fail
[ 1290.053499] sfp sfp1: los 0 -> 1
[ 1290.056725] sfp sfp1: SM: enter present:up:fail event los_high
[ 1290.062551] sfp sfp1: SM: exit present:up:fail

In the OpenWrt source, there is a series of patches in target/linux/generic/backport-6.6 and target/linux/generic/pending-6.6, which make changes to drivers/net/phy/realtek.c. At first, I suspected that was where the fixup got broken, so I tried removing all of the pending-6.6 patches related to realtek.c. That got rid of the -EOPNOTSUPP error, but the module still wasn't working at all.

Would someone be able to look into this issue? I'm willing to help with testing however I can.

rmandrad · December 17, 2024, 2:13pm

just some thoughts

is this driver identified as rtl8261n ? reason i am asking is you point to a driver patch

if so the code is under target/linux/mediatek/files-6.6/drivers/net/phy/rtl8261n

worth perhaps increase the kernel log (dynamic printk?)

perhaps you can try to use mdio to reset the device at boot ? or i2c tools

also under /sys/devices/platform/soc/15100000.ethernet/net/eth2/phydev/consumer:platform:15100000.ethernet/supplier/subsystem/drivers i see a number of drivers supported

check which directory is populated with something like this

./Aquantia AQR113C:
total 0
--w-------    1 root     root          4096 Dec 17 14:11 unbind
--w-------    1 root     root          4096 Dec 17 14:11 uevent
lrwxrwxrwx    1 root     root             0 Dec 17 14:11 module -> ../../../../module/aquantia
lrwxrwxrwx    1 root     root             0 Dec 17 14:11 i2c:sfp2:11 -> ../../../../devices/platform/sfp2/mdio_bus/i2c:sfp2/i2c:sfp2:11
lrwxrwxrwx    1 root     root             0 Dec 17 14:11 i2c:sfp1:11 -> ../../../../devices/platform/sfp1/mdio_bus/i2c:sfp1/i2c:sfp1:11

this will identify the actual driver that is being used

rmandrad · December 17, 2024, 3:58pm

assuming is a rtl8261 maybe worth reporting the issue to whom committed this https://git.openwrt.org/?p=openwrt/openwrt.git;a=commit;h=cfe8e6e75fed4d18f234334b96061fbbb7fd0c64

apparently is only for 10g

Betonmischer · December 17, 2024, 5:14pm

Thank you for the ideas. But I don't quite get why you are referring to rtl8261n. There must be some mixup. The OEM SFP-2.5G module has a RTL8221B-VB-CG phy. I'll try to be more clear.

The Kernel patch I linked (https://patchwork.kernel.org/project/netdevbpf/patch/20240303102848.164108-7-ericwouds@gmail.com/) adds code to drivers/net/phy/sfp.c, which, to my understanding, makes the Kernel talk to this SFP module via I2C for it to be recognized by the Realtek driver. Otherwise no Phy driver is engaged.

In turn, the Realtek driver (drivers/net/phy/realtek.c) has upstream patches that are intended to solve the initial bug with the module losing link after reboot (I checked, and the patch code is there).

After I applied the sfp.c patch, I2C probe messages started to pop up in dmesg with dynamic debugging enabled.

I forgot to take a copy, but when I removed all of the pending-6.6 patches to realtek.c, I saw the driver talking to the Phy and correctly identifying it as RTL8221B-VB-CG. So, the problem at least partially lies in drivers/net/phy/realtek.c being updated in such a way that it broke compatibility with the OEM SFP-2.5G module. Which no one noticed before because a) this is a janky module from Sinovoip; b) the driver can't talk to it unless there is a fixup in sfp.c; c) the fixup patch hasn't been merged into the Kernel since March.

I could, of course, leave everything as it is now. It's not a big deal to reset eth2 a bunch of times after a reboot. I just wanted to get a proper solution. But I can barely make sence of C code, so my idea was to ask first and tinker later.

Betonmischer · December 17, 2024, 11:04pm

I've opened a bug report about this: https://github.com/openwrt/openwrt/issues/17295

bublikOff · December 18, 2024, 11:50am

Not a big expert in this topic but u can try to rename model to SFP-2.5G-T using i2csfp

Betonmischer · December 18, 2024, 12:41pm

I forgot to mention that I tried a similar approach, but the other way around. I renamed OEM SFP-2.5G-T to OEM SFP-2.5G-T-R-RM in sfp.c so that it could use the existing quirk. Didn't help, hence the need for the fixup.

totkeks · December 18, 2024, 2:44pm

Side question: so the information printed by ethtool -i eth is not correct regarding the driver?

ethtool -i eth2
driver: mtk_soc_eth
version: 6.6.63

My SFP module uses a Realtek 9601D SoC, I assume that means the PHY would be Realtek as well and not the driver of the router SoC.

Betonmischer · December 18, 2024, 3:51pm

In my case, it also always was mtk_soc_eth regardless whether the Realtek driver could reach the Phy or not. One way to figure out the Phy driver would be:

Build with CONFIG_KERNEL_DYNAMIC_DEBUG=y.
Run echo "file drivers/net/* +p" > /sys/kernel/debug/dynamic_debug/control.
Check if anything new appears in dmesg after resetting eth2.

You could also take a look inside /sys/devices/platform/soc/15100000.ethernet/net/eth2 as Rmandrad suggested. That didn't show anything useful for me, though.