True on the 1G side but less so on the 10G side. I have an SFP-only 10G switch and could use a way to link to the 1G network.
So, generally, yes. But if the alternative is a 10G-BASET SFP then not so much.
Using a 1gbit rj45 sfp in the sfp+ socket ought to work. Those are cheap, I'd just try. Not all 10gbit Copper SFP seem to support 1gbit btw
1 Like
Wrong again. This was entirely my own fault.
I somehow managed to write the page address (0x7f) to the page register before ending up with the correct code. The SFP should have ignored the invalid value and recorded 0 instead according to spec, but it didn't. And to make things even worse: Both the page register and the vendor registers where the RollBall "password" is stored, are persistent over power resets. So when I finally got the code working, I would save and restore the invalid 0x7f page for every RollBall command, just like the mdio-i2c implemetation does. Assuming the old value was sane....
Anyway, ethtool -m lan8 hex on works fine after restoring the page to 0.
1 Like
It's actually much worse. Now that I have the phy driver running, I also have the access to the phy internal sensor in addition to the SFP+ sensor. And what I see is terrifying. Both are idling at around 50 °C when the link is down, which is OK given the high ambient temp. But a couple of seconds after bringing up a 10gig link:
root@OpenWrt:~# grep . /sys/class/hwmon/hwmon[1,2]/temp1_input
/sys/class/hwmon/hwmon1/temp1_input:57691
/sys/class/hwmon/hwmon2/temp1_input:85468
hwmon2 is the AQR113C internal sensor. And it stays around 85 °C. The driver defines the "critical" limit as 70 °C, so this is not good at all. I also have problems keeping the link up. Wouldn't be surprised if that's due to thermal shutdown in the phy.
Starting to think that this SFP+ is useless. Looks like it requires cooling the SFP cage to less than 20 °C just to keep it from reaching thermal shutdown all on its own. Got a tips for better/cooler SFP+s. Will get some of those.
2 Likes
Or maybe there's a reason the passively cooled sks8300 disappeared 
If it's DAC only then it should be ok, which makes sense if you plan in advance and get DAC capable NICs. Same is true on the NIC side - RJ45 10G NICs run much hotter than their SFP counterparts.
I've got a bunch of Mellanox connectX 3 and 4-lx NICs, they're very cheap on eBay and future proof with options for 25G or 40G in addition to 10G. The 10G connection to my servers is just a secondary link to carry to my desktop via cat6a, the other port on each connectx-3 NIC is a 40G infiniband link to a 16port infiniband IS5022 switch that are also super cheap now.
That works if all equipment lives in the same rack, less so if you want preexisting cat6 to distribute ethernet around the house... Although for now 2.5gbit would be enough there for me and those phys should be much cooler.
Can't imagine SR/LR is much of a problem either. SX certainly isn't.
10GBase-T with other phys is most likely fine as well. I gather these Aquantia chips are known for temperature issues. Guess they never should have been used in SFPs. There are better options now.
FWIW; the other end of my link is a cheap AQC107 based Asus card with a huge (compared to an SFP at least) passive cooler. And it also runs quite hot:
enp1s0-pci-0100
Adapter: PCI adapter
PHY Temperature: +70.1°C
MAC Temperature: +67.5°C
I actually have a AQC113CS PCIe card in my PC currently and it's not as hot but it does have a heatsink and the drivers are not 100% with some suspend/resume bugs.
The best thing about those cards is the ability to use a gen3 x2 or gen4 x1 slot.
enp6s0-pci-0600
Adapter: PCI adapter
PHY Temperature: +48.0°C
MAC Temperature: +48.0°C
So which phy are ok and how do we make sure they are used in a given SFP?
If you are on RTL930x, then you need to provide a two-cell interrupt (interrupt line + routing indication). This is different from the upstream driver which only has a one-cell interrupt spec.
I believe I only made one stupid mistake affecting clause 22, and that's fixed up and pushed now. Tested also the original code, and it didn't work as expected.
But with my fixes I can now talk to both these copper phys, using clause 22 for the first and RollBall for the second:
root@OpenWrt:~# dmesg|grep PHY
[ 74.461294] rtl83xx-switch switch@1b000000 lan5: PHY [smbus:sfp-p5:16] driver [Marvell 88E1111] (irq=POLL)
[ 95.836352] rtl83xx-switch switch@1b000000 lan8: PHY [smbus:sfp-p8:11] driver [Aquantia AQR113C] (irq=POLL)
Clause 22 access works:
root@OpenWrt:~# mdio smbus:sfp-p5
DEV PHY-ID LINK
0x16 0x01410cc2 down
root@OpenWrt:~# mdio smbus:sfp-p5 phy 22
BMCR(0x00): 0x1140
flags: -reset -loopback +aneg-enable -power-down -isolate -aneg-restart
-collision-test
speed: 1000-full
BMSR(0x01): 0x7949
capabilities: -100-t4 +100-tx-f +100-tx-h +10-t-f +10-t-h -100-t2-f -100-t2-h
flags: +ext-status -aneg-complete -remote-fault +aneg-capable -link
-jabber +ext-register
ID(0x02/0x03): 0x01410cc2
ESTATUS(0x0F): 0xf000
capabilities: +1000-x-f +1000-x-h +1000-t-f +1000-t-h
1 Like
Hmm? I don’t see a local one.
I can confirm that 10GBASE-T SFP+ with Aquantia PHY have temperature issues. I have three of those (bought from different sources) and they all go up to >80°C under load, as reported by their DOM, and >70°C when idle, in a non-AC'd environment. In addition, they're causing occasional retransmits under load, likely due to these temperature issues. I reckon you can only use these with active cooling and in an AC'd environment, like a DC.
You can recognize Aquantia-based ones before buying, they're usually advertised as "for cable lengths up to 30m". The positive side of these models is, they can link at 1G/2.5G/5G/10G speeds (while the switch always sees just 10G, at least for all three of mine).
For where I have to use 10GBASE-T, I switched to (much more expensive) ones advertised for "up to 80m". These link at 10G only, and run at least 10°C less hot, plus I saw zero retransmits under load. I can't currently figure out the PHY that's built into these, though, due to lack of an OpenWrt-capable switch with SFP+ ports, and because they're in active use (I refrained from buying a spare, due to their heavy price).
1 Like
AQR-s are crazy hot even when not in SFP-s, 70-80C is nothing for them basically.
I regularly see them go really high in regular routers as well.
1 Like
Educated guesses about 10gbase-T SFP+ modules based on my research so far:
- Broadcom BCM84891 - 80m 10GbaseT 1.8W (COOLEST)
- Marvell 88X3310, AQR113C - 30m 1/2.5/5/10GbaseT >2.5W (HOT)
- Realtek RTL8261N - 30m 1/2.5/5/10GbaseT <2.5W
I think I have a realtek SFP, it's hot but not the crazy temperature that others have reported.
1 Like
Shouldn't the RTL8261N ones be without 1GBE support? I don't have any, but most SFP+ advertised as Realtek, that I (or rather, Google) could find carry a note "no 1G support".
Maybe? I am just guessing here. The Broadcomm units definitely say only 10G support.
This is unexpected. The phy is supposed to support the whole range except 10 Mbps: https://www.broadcom.com/products/ethernet-connectivity/phy-and-poe/copper/10gbase-t-phy/bcm84891l
Maybe we need a quirk in the sfp driver for it to figure out that there is a phy in there?
YAY! Finally managed to dump the lplr (bootrom) from the rtl9303.
I only found a single weird thing in that code, before it jumps to 0x9F000000:
if (*(u32_t *) 0xB8001000 & 0x8000000)
{
*(u32_t *) [0xB800C000] = 0x9F00DFB0 & 0x1FFFFFFF;
*(u32_t *) [0xB800C004] = 0x9F00DFE0 & 0x1FFFFFFF;
*(u32_t *) [0xB800C00C] = 0x10D2;
*(u32_t *) [0xB800C008] = 2;
while ( !*(u32_t *) 0xB800C008 & 0x1000));
}
Also if you unset 0x40000 (FLASH_MAP1_DIS) in 0xB8001000 the spi nor will be mapped to 0x94000000.
note: fck this new user limit -.-