NC523SFP mini GBIC card (SFP+) with OpenWrt

Hello! I hope you are all well!

I decided to build my own router out of an old X86-64 PC and I have to admit, I am impress by the power OpenWrt has on this kind of hardware.

Everything works except for 1 thing. I have this NC523SFP card that has drivers in the Linux kernel.
With the generic X86-64 image, it is not recognized even when adding all Qlogic drivers.

And the reason was quite obvious.. According to this post, it requires the qlcnic and not the bnx2x driver.

No problem, I build my own image with the qlcnic and the sfp kernel modules.

I write the image to a USB stick and surprise, the card is recognize!
I then try to create interfaces based on them and OpenWrt let me do it.

Now I want to test if it works. I wire a fiber loop between the two ports and set one interface as a DHCP server and the other as a DHCP client.

Unfortunately, there is no communication. The LED’s don’t come up on the card even if I plug modules.

I have few experience with SFP’s, what could go wrong in my case?

Thank you for your help!

$ /sbin/modinfo -F firmware qlcnic 
phanfw.bin

(on a general purpose linux distribution) suggests that you might also need a binary firmware blob in addition to the kernel module, which doesn't appear to be packaged for OpenWrt (hint, this might be an opportunity to provide a patch).

https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain/phanfw.bin (--> /lib/firmware/phanfw.bin)

Is it hard to test it and propose a patch?

I have no experience unfortunately. I would need to read about it unless someone is willing to guide me. :nerd_face:

Testing, no, not at all. For starters, you literally just copy the downloaded file as /lib/firmware/phanfw.bin onto your router (scp, USB stick or similar) and reboot. The biggest potential issue with that would be the git webinterface or your browser messing up the file during the download, so do check the hashsum:

$ md5sum phanfw.bin
4143b2cb741455351a351b49467499b2  phanfw.bin

or

$ sha256sum phanfw.bin 
68a55514b067dda5a1d12b8f8b27ce44a0046486890d2c517a9402b9316c06fb  phanfw.bin

Once confirmed to be working, the packaging for OpenWrt isn't that hard either, just take aircard-pcmcia-firmware as an example and add yours in a similar way (as a new file, next to it, e.g. package/firmware/linux-firmware/netxen.mk, netxen-firmware, QLogic Intelligent Ethernet (3000 and 3100 Series) firmware):

1 Like

Nice! Thanks for showing me examples! Will do it once it works!

So I have added the file to /lib/firmware/qlcnic (I have created the qlcnic folder since it seemed to be mandatory when looking at the other blobs).

The md5 checksum match what you have provided.

Now, I’m not sure how to make it take into account. I tried restarting the associated interfaces but to no avail. I believe that if I restart, I will lose that patch unfortunately.

I have not seen any difference yet in the logs :

Mon Jan  8 07:11:37 2007 daemon.notice netifd: Interface 'Fibre2' is enabled
Mon Jan  8 07:11:46 2007 daemon.notice netifd: Interface 'Fibre1' is now down
Mon Jan  8 07:11:47 2007 daemon.notice netifd: Interface 'Fibre1' is disabled
Mon Jan  8 07:11:47 2007 kern.info kernel: [ 3933.969946] qlcnic 0000:02:00.0 eth1: Rx Context[0] Created, state 0x2
Mon Jan  8 07:11:47 2007 kern.info kernel: [ 3934.043253] qlcnic 0000:02:00.0 eth1: Tx Context[0x8000] Created, state 0x2
Mon Jan  8 07:11:47 2007 daemon.notice netifd: Interface 'Fibre1' is enabled
Mon Jan  8 07:11:47 2007 daemon.notice netifd: Interface 'Fibre1' is setting up now
Mon Jan  8 07:11:47 2007 daemon.notice netifd: Interface 'Fibre1' is now up
Mon Jan  8 07:11:47 2007 kern.info kernel: [ 3934.057479] qlcnic 0000:02:00.0 eth1: Tx Context[0x8008] Created, state 0x2

Nope, the module is searching for /lib/firmware/phanfw.bin, no sub-directory.

You haven't made any patch, yet (that's done at build time, inside the OpenWrt source). Just added a new (firmware-) file to your overlay(fs), it's retained over reboots.

Oh you are right, it stayed following the reboot. I should have taught of that since it can retain settings.

So I placed the blob at /lib/firmware/phanfw.bin and rebooted.

Unfortunately, no change.
Logs look similar :

Mon Jan  8 07:28:05 2007 kern.info kernel: [   14.557081] qlcnic 0000:02:00.0 eth1: Rx Context[0] Created, state 0x2
Mon Jan  8 07:28:05 2007 kern.info kernel: [   14.864728] qlcnic 0000:02:00.0 eth1: Tx Context[0x8000] Created, state 0x2
Mon Jan  8 07:28:05 2007 kern.info kernel: [   14.878955] qlcnic 0000:02:00.0 eth1: Tx Context[0x8008] Created, state 0x2
Mon Jan  8 07:28:05 2007 daemon.notice netifd: Interface 'Fibre1' is enabled
Mon Jan  8 07:28:05 2007 daemon.notice netifd: Interface 'Fibre1' is setting up now
Mon Jan  8 07:28:05 2007 daemon.notice netifd: Interface 'Fibre1' is now up
Mon Jan  8 07:28:06 2007 kern.info kernel: [   15.432053] qlcnic 0000:02:00.1 eth2: Rx Context[1] Created, state 0x2
Mon Jan  8 07:28:06 2007 kern.info kernel: [   15.890920] qlcnic 0000:02:00.1 eth2: Tx Context[0x8001] Created, state 0x2
Mon Jan  8 07:28:06 2007 kern.info kernel: [   15.905146] qlcnic 0000:02:00.1 eth2: Tx Context[0x8009] Created, state 0x2
Mon Jan  8 07:28:06 2007 daemon.notice netifd: Interface 'Fibre2' is enabled

I have no personal experience with this hardware, the missing firmware was just an obvious (potential) omission.

1 Like

dmesg might provide further clues, have a look at ethtool, check what happens (error messages and dmesg again), if you use iproute2 (ip) to set up the link, tcpdump, etc.

Also have a quick test with a general purpose linux distribution of your choice (e.g. Debian, Fedora, arch, Gentoo, Mageia/ Mandriva, OpenSuSE, Ubuntu, etc.) - does it work, what's needed there (dmesg, ls -1 /sys/module/, lspci -knn).

1 Like

Alright, I tested a bunch of stuff..

I went straight ahead with Fedora 37 and Ubuntu 18.04.6, especially Ubuntu since this article claim it is working out of the box with it.
Yet, unfortunately, both distributions claimed to not have any cable attached to them but saw the card and loaded the driver correctly.

They also both contain the firmware blob.

Wanting to test the modules :
ethtool --module-info with both interfaces return :
netlink error : Openration not supported.

Link detected: no
[liveuser@localhost-live ~]$ ethtool enp2s0f0
Settings for enp2s0f0:
Supported ports: [ TP FIBRE ]
Supported link modes:
10000baseT /Full
Supported pause frame use: No
Supports auto-negotiation: No
Supported FEC modes: Not reported
Advertised link modes:
10000baseT/Full
Advertised pause frame use: No
Advertised auto-negotiation: No
Advertised FEC modes: Not reported
Speed: Unknown!
Duplex: Unknown! (255)
Auto-negotiation: off
Port: Other
PHYAD: O
Transceiver: internal
netlink error: Operation not permitted
Current message level: 0x00000000 (0)
Link detected: no
[liveuser@localhost-live ~]$ ethtool enp2s0f
enp2s0f0
enp2s0f1
[liveuser@localhost-live ~]$ ethtool -m enp2s0f1
netlink error: Operation not permitted
[liveuser@localhost-live ~]$ sudo ethtool -m enp2s0f1
netlink error: Operation not supported
[liveuser@localhost-live ~]$ ethtool --module-info enp2s0f1
netlink error: Operation not permitted
[liveuser@localhost-live~]$

At any time, I have not seen any lights appear on the card which worries me.

For now I have 3 options :

  • Later today I can try the other module I have in the home (that I want to use with the card). It is the Gpon of my internet provider. I will only sync at 1 Gb/s, not the 2.5 Gb/s but my plan is only 500/500 Mb/s anyway for now.
  • I purchase another card that is known to have good support already with OpenWrt and does not complain about the transceiver. (Might support the Gpon or not, it is a risk). Cheaper but might not work.
  • I scrap all of that and go for a X86-64 mini computer with some I225-V (2.5 Gb/s) and a 2.5 to 2.5 Gb/s media converter. This option is more expensive but I know it will work.

What are your taught about the setup I'm trying to accomplish?
Ultimately perhaps I'm seeing bigger than my needs. All I want is OpenWrt end-to-end, avoid my ISP horrible modem and connect the server and the access point. Space is limited so everything else is wireless.

After some thinking, I’m wondering if the card is even compatible with 1 Gb/s transceivers.