ER-X-SFP - SFP (eth5) port has link-state (LED = lit), but swconfig says link: port:5 link:down

Hi,

I think PHYLIB is the problem. PHYLIB knows that at8033 exits but doesn't have a driver.
You can enable the driver but that is not going to work because at8033 current driver doesn't understand the SFP cage.
But what PHYLIB does, it puts the PHY is auto negotiation mode.
But that doesn't work well with your SGMII sfp module.

What you can try is same phytool write command.
This disables the auto negotiation on the at8033 PHY.
Then take the SFP plug out and re-insert the SFP module.
So it is in a know state.
This should work.

Other option also put the PHY inside the SFP in forced mode.

i2cset -y 0 0x56 0x00 0x0140 w

I run the phytool command

phytool write eth0/7/0 0x0140

and then take the SFP plug out and re-insert the SFP module.
and it works.

We need more improvement.

Hi,

I redo some detection code and removed the phy-node in dts.
Now PHYLIB doesn't touch the phy anymore.

Can you please try this branch?

1 Like

thanks, I will try it latter.

I try the new branch and it work. but...
This time I don't need the phytool command.
And to get it work I have to plug out and re-insert the SFP module.

So it doesn't work directly? You first have to replug the sfp module?
What happens after a power cycle?

everty time the device reboot, I need to do a replug-the-sfp-module action to get it work. (only) after that, unplugging the cable and plugging in again is also working.

I've tested the current solution on two ER-X-SFP with each equipped UF-SM-1G-S transceiver. Everything worked flawlessly. Also after reboot, unplug/replug, remove/reinsert, network restart etc.

We created a backport for the current openwrt 19.07 branch (rc2 state / git), for your reference:
https://dev.opennet-initiative.de/browser/on_firmware/patches/edgerouterx-sfp.patch

As know the status of the link and SFP is not known (always up). Any plans how to proceed and bringing it upstream to openwrt (trunk or even 19.07)? Or any other tasks to be done first / cleanup?

If you need further tests or support I'm (we are) happy to follow up here or in person at 36c3.

3 Likes

Hi, i have an ER-X-SFP running openwrt 19.07.01 and I'd like to use it with my Technicolor ONU.
It works out of the box using edgeOS, so it should be easy to let it work.
I followed a bit the thread, but I haven't understood if there's a way to use it and if it has been merged upstream

Edit: I took the patch published by @mathiashr, adapted it to run on openwrt 19.07 RC2 and compiled it. Unfortunately when i plug my sfp module the link state remains down.
I also tryied to force it up with the phytool command provided above, but nothing changes.

Hi @gabri94, can you explain what kind of SFP exactly the "Technicolor ONU" is? Do you have other SFPs available to test? I've used Cisco and UBNT which worked well.

Also please kindly understand that we still maintain the above patch against the OpenWrt 19.07 branch (currently at .02). We run compile tests, last functional test was at RC2 as stated above. Would be great if you can evaluate further.

Hi, I am trying to apply the patch to the commit with hash "c56ed72d2bc6dbee3ff82b4bd42e1768f1a2c737"
which should corresponds to the head of openwrt-19.07.

The error i get is:

edgerouterx-sfp.patch:93: trailing whitespace.
	
edgerouterx-sfp.patch:306: trailing whitespace.
		;;
error: corrupted patch at line 307

I circumvented it by removing that block and doing the changes by hand.

Regarding to the SFP, it is a Telecom Italia ONT for GPON FTTH access. Unfortunately I haven't got others SFP to try, but I tested this SFP with the latest version of EdgeOS (v1) and it works out of the box.

Good Evening,

updated our patch to remove the trailing whitespaces (was already included in the upstream patchset). The currupted error is not visible here - patch and unpatch was successfull on our codebase.

Can you retest the patch and/or investigate further about the reason why you can't apply the patch?

Have a nice day!

Patch patches/edgerouterx-sfp.patch
patching file openwrt/target/linux/ramips/files-4.14/drivers/net/ethernet/mediatek/gsw_mt7620.h
patching file openwrt/target/linux/ramips/files-4.14/drivers/net/ethernet/mediatek/gsw_mt7621.c
patching file openwrt/target/linux/ramips/dts/mt7621.dtsi
patching file openwrt/target/linux/ramips/base-files/etc/board.d/02_network
Hunk #1 succeeded at 249 (offset -1 lines).
Hunk #2 succeeded at 259 (offset -1 lines).
patching file openwrt/target/linux/ramips/dts/UBNT-ERX-SFP.dts
patching file openwrt/package/network/utils/phytool/Makefile

I'm not getting the trailing whitespaces messages anymore (though i think they were warning)
but I'm stil getting the last one:

error: corrupted patch at line 307

At this point i don't know could be.
I am applying the patch using the command:

git apply edgerouterx-sfp.patch

on the commit hash:

c56ed72d2bc6dbee3ff82b4bd42e1768f1a2c737

Can you confirm this is consistent with your patch?

I have fixed the patch. You can find my version here https://pastebin.com/wKNfMqsG

This is applicable on the openwrt feed using that command.
I had to remove "openwrt/" from all the path, it looks like you were applying the patch from outside the repo.

Now, I'd like to help you troubleshoot why my SFP is not working with this patch. Is there some log or some command which would help you?

Ok, understood. We are using Quilt (https://wiki.debian.org/UsingQuilt) in our automated build environment, therefor the path is different than using "git apply".

Tools used here and also referred in forum threads:

  • Link status: phytool print eth0/7 ("flags +aneg-complete +link" = link is active)
  • Transceiver modell und serial number: i2cdump -y 0 0x50 (via installed package "i2c-tools")

Hope this helps to get one step forwards. Also you should consider checking kernel.org for driver support of your transceiver.

Dengqf6 is pushing openwrt to v5.4 kernel. Which support the PHYLINK API, DSA and port 5 of the switch also my AT803x driver with SFP support.
So the linux kernel read the SFP module and sets-up the mac-phy chain in the right mode.

Note: v5.4 has issues with NAND driver/chip. Depending on the NAND flash chip, you may get ECC errors and can't flash it.

Also AT803x PHY which is connected to SFP cage only supports 1000BASE-X. SGMII may work but only in FORCED 1GBIT Full duplex mode. But 10/100MBIT will not work!

The kernel will report your SFP like this.

[   69.143018] sfp sfp_eth5: module FS               SFP-GE-BX        rev      sn <snip>      dc 190803
[   69.178744] Atheros 8031 ethernet mdio-bus:07: SFP interface 1000base-x

also ethtool -m eth5 gives you full information about your SFP module.

root@OpenWrt:/# ethtool -m eth5
        Identifier                                : 0x03 (SFP)
        Extended identifier                       : 0x04 (GBIC/SFP defined by 2-wire interface ID)
        Connector                                 : 0x07 (LC)
        Transceiver codes                         : 0x00 0x00 0x00 0x40 0x00 0x00 0x00 0x00 0x00
        Transceiver type                          : Ethernet: BASE-BX10
        Encoding                                  : 0x01 (8B/10B)
        BR, Nominal                               : 1300MBd
        Rate identifier                           : 0x00 (unspecified)
        Length (SMF,km)                           : 10km
        Length (SMF)                              : 10000m
        Length (50um)                             : 0m
        Length (62.5um)                           : 0m
        Length (Copper)                           : 0m
        Length (OM3)                              : 0m
        Laser wavelength                          : 1550nm
        Vendor name                               : FS
        Vendor OUI                                : 00:00:00
        Vendor PN                                 : SFP-GE-BX
        Vendor rev                                :
        Option values                             : 0x20 0x0a
        Option                                    : RX_LOS implemented
        Option                                    : TX_FAULT implemented
        Option                                    : Power level 3 requirement
        BR margin, max                            : 0%
        BR margin, min                            : 0%
        Vendor SN                                 : <snip>
        Date code                                 : 190803
        Optical diagnostics support               : Yes
        Laser bias current                        : 35.936 mA
        Laser output power                        : 0.2490 mW / -6.04 dBm
        Receiver signal average optical power     : 0.2130 mW / -6.72 dBm
        Module temperature                        : 51.18 degrees C / 124.12 degrees F
        Module voltage                            : 3.2102 V
        Alarm/warning flags implemented           : Yes
        Laser bias current high alarm             : Off
        Laser bias current low alarm              : Off
        Laser bias current high warning           : Off
        Laser bias current low warning            : Off
        Laser output power high alarm             : Off
        Laser output power low alarm              : Off
        Laser output power high warning           : Off
        Laser output power low warning            : Off
        Module temperature high alarm             : Off
        Module temperature low alarm              : Off
        Module temperature high warning           : Off
        Module temperature low warning            : Off
        Module voltage high alarm                 : Off
        Module voltage low alarm                  : Off
        Module voltage high warning               : Off
        Module voltage low warning                : Off
        Laser rx power high alarm                 : Off
        Laser rx power low alarm                  : Off
        Laser rx power high warning               : Off
        Laser rx power low warning                : Off
        Laser bias current high alarm threshold   : 100.000 mA
        Laser bias current low alarm threshold    : 0.000 mA
        Laser bias current high warning threshold : 95.000 mA
        Laser bias current low warning threshold  : 0.000 mA
        Laser output power high alarm threshold   : 0.7079 mW / -1.50 dBm
        Laser output power low alarm threshold    : 0.0891 mW / -10.50 dBm
        Laser output power high warning threshold : 0.6310 mW / -2.00 dBm
        Laser output power low warning threshold  : 0.1000 mW / -10.00 dBm
        Module temperature high alarm threshold   : 90.00 degrees C / 194.00 degrees F
        Module temperature low alarm threshold    : -45.00 degrees C / -49.00 degrees F
        Module temperature high warning threshold : 85.00 degrees C / 185.00 degrees F
        Module temperature low warning threshold  : -40.00 degrees C / -40.00 degrees F
        Module voltage high alarm threshold       : 3.7950 V
        Module voltage low alarm threshold        : 2.8050 V
        Module voltage high warning threshold     : 3.4650 V
        Module voltage low warning threshold      : 3.1350 V
        Laser rx power high alarm threshold       : 0.7079 mW / -1.50 dBm
        Laser rx power low alarm threshold        : 0.0035 mW / -24.56 dBm
        Laser rx power high warning threshold     : 0.6310 mW / -2.00 dBm
        Laser rx power low warning threshold      : 0.0040 mW / -23.98 dBm

Hello @Thirsty,

I have built a freifunk-firmware based on the latest snapshot ( 7daab6286110b95167e291409395494f18edc9dc) and I am not seeing an eth5. In the source tree for the ERX-SFP I do not see a CONFIG_AT803X_PHY=y option in https://github.com/openwrt/openwrt/blob/master/target/linux/ramips/mt7621/config-5.4 either.

Is this something which should be added? Is there some other trick I would need to do to get the SFP port active?

Thanks in advance,

This is my latest PR https://github.com/openwrt/openwrt/pull/2901
To get eth5 you have to make changes in the DTS file of the device.
See PR how to do that and also the other bits en pieces.

BTW: NAND issue is fixed so you can build a full image with v5.4 kernel and flash it to nand.
Also John is working on HWNAT for mt7621. He updated HWNAT for mt7623 this week.

1 Like

I built this branch yesterday and can confirm that the SFP port was working. I also left a comment here (https://github.com/vDorst/openwrt/commit/70188cf0837e7991a8eea390fbd75d89062941b3#commitcomment-38365000).

I will also put a comment into the PR.

Unfortunately I can not test the device under load because VLAN's don't seem to be working correctly. But that is off-topic from this thread, so I believe I will make a new one. I will link the new thread here once I have made it.

Thanks again for the good work.

Here is the link to the VLAN topic...