Eth4 Link stays down after unplugging and replugging cable

I'm using a R86S and I have installed the most recent snapshot on it.I have the model that includes a Mellanox CX3 with 2 SFP ports.

The drivers are installed and the connections both work with my Mikrotik SFP modules. eth3 is my WAN and eth4 is on LAN. If I unplug from my eth3 WAN and plug back in the link will go down then come back once once the cable is plugged back in.

If I do the same thing eth4 once I unplug the cable the link stays down even after reconnecting the cable. The device on the other end recognizes a cable is connected but fails to pull an address as OpenWRT registers the link as down. This also happens if the device is powered down and I power it back on. The link is only reestablishing once i reboot OpenWRT.

Anyone have any ideas?

1 Like

Is eth4 a SFP?

Yes, eth4 and eth3 are both the same MikroTik S+RJ10 model.

I have also used a DAC cable as well instead of a SFP on eth4 and the results are the same.

Since it works if you boot with hardware connected, that probably mean that the sfp ports on your device is only specified as a network connectors in the hardware device tree file.
So this means that the sfp will work as static ports if something is connected to them at boot time but they are not hot-pluggable.

It isn’t necessarily that easy to fix this problem, it can be lack of hardware probing for the dts file or problem with some drivers.

Do the port get any initialization error or no text at all in the system log when doing a hot-plugging?

The other port on this card works fine with hotplug. I have swapped them as well. When the port is set as my WAN port hotplug works fine. Just when the port is added to LAN hotlplug does not work. I have also uses different branded SFP adapters from Wiitek that had the same result. I also bypasses using a SFP adapter and used a DAC cable which had the same results.

The only message displayed is the link going down. There is no other output after that even when unplugging and replugging.

Are they ethernet or SFP ports?
This hot plug problem is only for SFP, and it can be for one single SFP, it can work with only one SFP connected, it can stop working if you connect two SFP.
This is hard to get working and I have tried on the d-link switches.

The thing is that OpenWrt isn’t that stable for managing SFP and this system is super complex without any real standard solution. Every manufacturer makes it own hardware solutions for controlling the ports.
Mostly because most home routers don’t have those so there isn’t much experience.

It really sounds like standard output of hot plugging not configured at all or configured but not working in the dts files.

Both ports are SFP ports.

The first thing you should do if the device wiki doesn’t say anything about this is to find the commit message for that device (in table of hardware) and see if it say anything about known sfp bugs or if there is a forum tread for the implementation of the device it could have some clues of the status of sfp or find the person that included the device to OpenWrt.

Or find the dts file for that device and look.

What is the exact path name to the download of your image?

I don't have the download link saved. I used the firmware selector for Generic x86/x64 on Snapshot r20426-314cad2cba . I added the additional packages to the install image. It includes the drivers needed for the Mellanox along with a few extras I planned on using.

kmod-ptp kmod-mlx4-core kmod-mmc-spi kmod-pps kmod-mmc kmod-lib-crc-itu-t kmod-lib-crc7 kmod-macvlan luci luci-theme-openwrt-2020 kmod-sched-cake luci-app-sqm luci-app-upnp

I am not sure how it works with generic x86/x64 computers and openwrt but they don’t have dts files in the source code.

But as usual with Linux and hardware, nothing happens by itself and these sfp ports and the communication for these needs to be set up somewhere if you want hot pluggable ports.

Also have the r86s; it seems to only be an issue when interfaces are bridged; removing them from the bridge seems to let them work fine.

NIC setting fixed speed, works for me.

ethtool -s eth3 speed 10000 autoneg off
1 Like

I'm running into the exact same problem: https://forum.openwrt.org/t/eth4-link-stays-down-after-unplugging-and-replugging-cable

I don't think it's a hardware problem, as it properly accepts hotplugging on Ubuntu(although it takes a bit longer).

It does not seem to matter whether the port belongs to a bridge or not. The same problem occurs even if it is designated as a stand-alone interface.

Changing the auto-negotiation setting will restart the port. This will have an effect but only once.

Never found a working solution on the SW side. I ended up connecting it to an always on switch.

I found that this problem does not seem to occur in the snapshot version (r22135-530f5c2fda, 2023-02-25 10:24:59).
Kernel version is newer than the stable releases and probably includes a workaround for this problem.

Did you add this command to Startup or did you run it over ssh?

try