Hi,
this should do the trick, just checkout this pull-request and make sure you compile the TESTING_KERNEL (linux 5.10):
try this patch:
https://patchwork.ozlabs.org/project/openwrt/patch/20210930171541.13804-1-fercerpav@gmail.com/
This patch fixes the same issue of RTL8231 on my ApresiaLightGS120GT-SS (RTL8382M, 16 + 4 combo)
1 Like
Careful that you apply this patch to the 5.10 RTL8231 driver (it is patching the 5.4 one), the PR works with the 5.10 port made by @musashino.
1 Like
Thanks very much, with that and the patch from @musashino everything seems to work now, even the SFP slots!
For reference, this is my current device tree file, the two SFP slots are directly attached to the main chip on pins 94-97 and 99-102, as described in Section 6.5.5 of the datasheet, so I presume that phy-mode = "1000base-x"; is correct. It works in any case. The I2C connections I traced out are identical to those recently posted for the DGS-1210-10P H/W:R1, which is encouraging. I couldn't find a connection to the TX-fault pin of the SFP slots to any of the pins of the RTL8231.
rtl8382_d-link_dgs-1210-26.dts:
// SPDX-License-Identifier: GPL-2.0-or-later OR MIT
#include "rtl8382_d-link_dgs-1210.dtsi"
/ {
compatible = "d-link,dgs-1210-26", "realtek,rtl838x-soc";
model = "D-Link DGS-1210-26";
/* Left SFP slot, port 25 */
i2c0: i2c-gpio-0 {
compatible = "i2c-gpio";
sda-gpios = <&gpio1 6 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
scl-gpios = <&gpio1 7 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
i2c-gpio,delay-us = <2>;
#address-cells = <1>;
#size-cells = <0>;
};
sfp0: sfp-p25 {
compatible = "sff,sfp";
i2c-bus = <&i2c0>;
los-gpio = <&gpio1 9 GPIO_ACTIVE_HIGH>;
mod-def0-gpio = <&gpio1 8 GPIO_ACTIVE_LOW>;
tx-disable-gpio = <&gpio1 11 GPIO_ACTIVE_HIGH>;
};
/* Right SFP slot, port 26 */
i2c1: i2c-gpio-1 {
compatible = "i2c-gpio";
sda-gpios = <&gpio1 1 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
scl-gpios = <&gpio1 2 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
i2c-gpio,delay-us = <2>;
#address-cells = <1>;
#size-cells = <0>;
};
sfp1: sfp-p26 {
compatible = "sff,sfp";
i2c-bus = <&i2c1>;
los-gpio = <&gpio1 4 GPIO_ACTIVE_HIGH>;
mod-def0-gpio = <&gpio1 3 GPIO_ACTIVE_LOW>;
tx-disable-gpio = <&gpio1 12 GPIO_ACTIVE_HIGH>;
};
};
&gpio1 {
status = "okay";
};
ðernet0 {
mdio: mdio-bus {
compatible = "realtek,rtl838x-mdio";
regmap = <ðernet0>;
#address-cells = <1>;
#size-cells = <0>;
EXTERNAL_PHY(0)
EXTERNAL_PHY(1)
EXTERNAL_PHY(2)
EXTERNAL_PHY(3)
EXTERNAL_PHY(4)
EXTERNAL_PHY(5)
EXTERNAL_PHY(6)
EXTERNAL_PHY(7)
INTERNAL_PHY(8)
INTERNAL_PHY(9)
INTERNAL_PHY(10)
INTERNAL_PHY(11)
INTERNAL_PHY(12)
INTERNAL_PHY(13)
INTERNAL_PHY(14)
INTERNAL_PHY(15)
EXTERNAL_PHY(16)
EXTERNAL_PHY(17)
EXTERNAL_PHY(18)
EXTERNAL_PHY(19)
EXTERNAL_PHY(20)
EXTERNAL_PHY(21)
EXTERNAL_PHY(22)
EXTERNAL_PHY(23)
INTERNAL_PHY(24)
INTERNAL_PHY(26)
};
};
&switch0 {
ports {
#address-cells = <1>;
#size-cells = <0>;
SWITCH_PORT(0, 1, qsgmii)
SWITCH_PORT(1, 2, qsgmii)
SWITCH_PORT(2, 3, qsgmii)
SWITCH_PORT(3, 4, qsgmii)
SWITCH_PORT(4, 5, qsgmii)
SWITCH_PORT(5, 6, qsgmii)
SWITCH_PORT(6, 7, qsgmii)
SWITCH_PORT(7, 8, qsgmii)
SWITCH_PORT(8, 9, internal)
SWITCH_PORT(9, 10, internal)
SWITCH_PORT(10, 11, internal)
SWITCH_PORT(11, 12, internal)
SWITCH_PORT(12, 13, internal)
SWITCH_PORT(13, 14, internal)
SWITCH_PORT(14, 15, internal)
SWITCH_PORT(15, 16, internal)
SWITCH_PORT(16, 17, qsgmii)
SWITCH_PORT(17, 18, qsgmii)
SWITCH_PORT(18, 19, qsgmii)
SWITCH_PORT(19, 20, qsgmii)
SWITCH_PORT(20, 21, qsgmii)
SWITCH_PORT(21, 22, qsgmii)
SWITCH_PORT(22, 23, qsgmii)
SWITCH_PORT(23, 24, qsgmii)
port@24 {
reg = <24>;
label = "lan25";
phy-mode = "1000base-x";
managed = "in-band-status";
sfp = <&sfp0>;
};
port@26 {
reg = <26>;
label = "lan26";
phy-mode = "1000base-x";
managed = "in-band-status";
sfp = <&sfp1>;
};
port@28 {
ethernet = <ðernet0>;
reg = <28>;
phy-mode = "internal";
fixed-link {
speed = <1000>;
full-duplex;
};
};
};
};
I believe this model might be able to use all 24 copper ports as well as the 2 SFP slots at the same time (as per the diagram in section 3.4 of the datasheet) - but I have not tried this yet.
3 Likes
Even in ApresiaLightGS120GT-SS, TX-Fault lines are not connected.
RTL8231 GPIO pin assignment:
0
1 SFP P18 I2C (SDA)
2 SFP P18 I2C (SCL)
3 SFP P18 MOD0
4 SFP P18 LOS
5
6 SFP P17 I2C (SDA)
7 SFP P17 I2C (SCL)
8 SFP P17 MOD0
9 SFP P17 LOS
10 LED "LOOP"
11 SFP P20 I2C (SDA)
12 SFP P20 I2C (SCL)
13 SFP P20 MOD0
14 SFP P20 LOS
15
16
17
18
19
20
21
22 SFP P19 I2C (SDA)
23 SFP P19 I2C (SCL)
24 SFP P19 MOD0
25 SFP P19 LOS
26
27
28
29
30
31
32
33 Button "RESET"
34 SYS SoC RESET?
35
36 LED unused LED (LED chip is soldered, but no hole on the case)
Note:
- 4x TX-Disable lines are provided by RTL8214FC
- this switch seems to be almost the same as D-Link DGS-1210-16 Rev.G
1 Like
Okay, almost: reset no longer works, which it did in a master from last week or so. I get the same console message as for the working case,
[ 909.732969] PLL control register: efffffff, applying reset value efffffff
but then it stays there and I have to power-cycle.
I recognized it as a potential issue in 5.10, but until now there was no report from anyone other than me and I couldn't determine if it was my environment-specific issue...
Oh, no, the reset bug on the 838x platform is back. This is bad, because it depends on the device you are using. The strange PLL message is there to help you understand what is going on.
What is happening is the following: The Flash on this platform can be addressed in a 3 or with a 4 byte wide mode. It depends on the type of flash being used, in particular its size. The flash driver once Linux is running can basically do whatever it wants, but U-Boot expects a certain state when it boots up and tries to read the flash. Interestingly, the reset does not reset the 3/4-byte mode and PLL upon reset (there are other things you can mess up which will prevent the device from booting even after the reset, I have found quite a few by accident). So you see this reset and then nothing happens any longer, because the reset happened, it is only u-boot that gets stuck and cannot boot.
The original flash driver carefully tried to avoid this effect together with the reset code. Because it depends on the device, you can easily fix it for one type and break it for another. It took a lot of time and effort from several developers to find a solution that actually worked on all devices. The new flash driver looks suspiciously free of the 3/4 byte hacks that were being used...
2 Likes
(Nearly?) all switches I've worked with until now have a GPIO pin (usually from an RTL8231, but can also be an internal one) wired to the SoC's hard reset pin. This GPIO can be used with the gpio-restart driver, and works reliably in my experience. This could help avoiding the current hacks.
2 Likes
I did some tests with 5.10:
Panasonic Switch-M8eG PN28080K (RTL8380M, 8 + 1 sfp)
Flash: MX25L25635F
current master
BusyBox v1.34.0 (2021-09-29 11:36:23 UTC) built-in shell (ash)
_______ ________ __
| |.-----.-----.-----.| | | |.----.| |_
| - || _ | -__| || | | || _|| _|
|_______|| __|_____|__|__||________||__| |____|
|__| W I R E L E S S F R E E D O M
-----------------------------------------------------
OpenWrt SNAPSHOT, r17668-6b4fa00317
-----------------------------------------------------
=== WARNING! =====================================
There is no root password defined on this device!
Use the "passwd" command to set up a new password
in order to prevent unauthorized SSH logins.
--------------------------------------------------
root@OpenWrt:/# devmem 0x1b000ff8
0x0000000F
root@OpenWrt:/tmp# reboot
root@OpenWrt:/tmp# [ 351.064036] device switch left promiscuous mode
[ 351.966128] switch: port 5(lan5) entered disabled state
...
[ 354.088495] device lan8 left promiscuous mode
[ 354.146298] switch: port 8(lan8) entered disabled state
[ 356.249143] reboot: Restarting system
[ 356.297476] System restart.
[ 356.334214] PLL control register: f, applying reset value efffffff
(stuck)
disable restoring pll reset value
BusyBox v1.34.0 (2021-09-29 11:36:23 UTC) built-in shell (ash)
_______ ________ __
| |.-----.-----.-----.| | | |.----.| |_
| - || _ | -__| || | | || _|| _|
|_______|| __|_____|__|__||________||__| |____|
|__| W I R E L E S S F R E E D O M
-----------------------------------------------------
OpenWrt SNAPSHOT, r17668-6b4fa00317
-----------------------------------------------------
=== WARNING! =====================================
There is no root password defined on this device!
Use the "passwd" command to set up a new password
in order to prevent unauthorized SSH logins.
--------------------------------------------------
root@OpenWrt:/# devmem 0x1b000ff8
0x0000000F
root@OpenWrt:/tmp# reboot
root@OpenWrt:/tmp# [ 351.064036] device switch left promiscuous mode
[ 351.966128] switch: port 5(lan5) entered disabled state
[ 352.230914] device lan1 left promiscuous mode
[ 352.288754] switch: port 1(lan1) entered disabled state
[ 352.643363] device lan2 left promiscuous mode
[ 352.701220] switch: port 2(lan2) entered disabled state
[ 353.237457] device lan3 left promiscuous mode
[ 353.295279] switch: port 3(lan3) entered disabled state
[ 353.379522] device lan4 left promiscuous mode
[ 353.437359] switch: port 4(lan4) entered disabled state
[ 353.520759] device lan5 left promiscuous mode
[ 353.578602] switch: port 5(lan5) entered disabled state
[ 353.659985] rtl83xx-switch switch@1b000000 lan5: Link is Down
[ 353.735740] RTL8380 Link change: status: 1, ports 1000
[ 353.805556] device lan6 left promiscuous mode
[ 353.863374] switch: port 6(lan6) entered disabled state
[ 353.947058] device lan7 left promiscuous mode
[ 354.004852] switch: port 7(lan7) entered disabled state
[ 354.088495] device lan8 left promiscuous mode
[ 354.146298] switch: port 8(lan8) entered disabled state
[ 356.249143] reboot: Restarting system
[ 356.297476] System restart.
[ 356.334214] PLL control register: f, applying reset value efffffff
U-Boot 2011.12.pre45842-1.00.22-svn5720 (Mar 12 2015 - 19:04:09)
Board: RTL838x CPU:500MHz LXB:200MHz MEM:300MHz
DRAM: 128 MB
SPI-F: 1x32 MB
...
APRESIA ApresiaLightGS120GT-SS (RTL8382M, 16 + 4 combo)
Flash: MX25L25635F
current master / disable restoring pll reset value
BusyBox v1.34.0 (2021-09-29 11:36:23 UTC) built-in shell (ash)
_______ ________ __
| |.-----.-----.-----.| | | |.----.| |_
| - || _ | -__| || | | || _|| _|
|_______|| __|_____|__|__||________||__| |____|
|__| W I R E L E S S F R E E D O M
-----------------------------------------------------
OpenWrt SNAPSHOT, r17664-8f5ee58e4a
-----------------------------------------------------
=== WARNING! =====================================
There is no root password defined on this device!
Use the "passwd" command to set up a new password
in order to prevent unauthorized SSH logins.
--------------------------------------------------
root@OpenWrt:/# devmem 0x1b000ff8 32
0xEFFFFFFF
root@OpenWrt:/# reboot
root@OpenWrt:/# [ 142.612286] device switch left promiscuous mode
[ 143.495717] switch: port 20(lan9) entered disabled state
...
[ 144.921043] device lan9 left promiscuous mode
[ 144.926393] switch: port 20(lan9) entered disabled state
[ 144.934678] in rtl838x_eth_stop
[ 144.938429] rtl838x-eth 1b00a300.ethernet eth0: Link is Down
[ 145.460657] rtl83xx-switch switch@1b000000 lan9: Link is Down
[ 145.467183] RTL8380 Link change: status: 1, ports 100
[ 147.727787] reboot: Restarting system
[ 147.731908] System restart.
[ 147.735022] PLL control register: efffffff, applying reset value efffffff
(stuck)
disable restoring pll reset value + set BIT(30)=0 in 0x1B000FF8
BusyBox v1.34.0 (2021-09-29 11:36:23 UTC) built-in shell (ash)
_______ ________ __
| |.-----.-----.-----.| | | |.----.| |_
| - || _ | -__| || | | || _|| _|
|_______|| __|_____|__|__||________||__| |____|
|__| W I R E L E S S F R E E D O M
-----------------------------------------------------
OpenWrt SNAPSHOT, r17665-bb2cf2b4f7
-----------------------------------------------------
=== WARNING! =====================================
There is no root password defined on this device!
Use the "passwd" command to set up a new password
in order to prevent unauthorized SSH logins.
--------------------------------------------------
root@OpenWrt:/# devmem 0x1b000ff8 32
0xEFFFFFFF
root@OpenWrt:/# devmem 0x1b000ff8 32 0xAFFFFFFF
root@OpenWrt:/# devmem 0x1b000ff8 32
0xAFFFFFFF
root@OpenWrt:/# reboot
root@OpenWrt:/# [ 514.782439] device switch left promiscuous mode
[ 515.650467] switch: port 20(lan9) entered disabled state
...
[ 517.092571] device lan9 left promiscuous mode
[ 517.097924] switch: port 20(lan9) entered disabled state
[ 517.106389] in rtl838x_eth_stop
[ 517.110059] rtl838x-eth 1b00a300.ethernet eth0: Link is Down
[ 517.632361] rtl83xx-switch switch@1b000000 lan9: Link is Down
[ 517.638887] RTL8380 Link change: status: 1, ports 100
[ 519.897804] reboot: Restarting system
[ 519.901924] System restart.
[ 519.905038] PLL control register: afffffff, applying reset value efffffff
U-Boot 2011.12.(2.1.5.67086)-Candidate1 (Apr 21 2020 - 16:39:05)
1.00.013
Board: RTL838x CPU:500MHz LXB:200MHz MEM:300MHz
DRAM: 256 MB
SPI-F: 1x32 MB
...
The address mode of the flash controller may be reset by the kernel before system reset.
BTW, Panasonic Switch-M*eG PN28xx0K series also have a GPIO pin on PCA9539 for hard-reset.
Yes, indeed, on the 838x platform using a GPIO to make a hard reset is the preferred way of performing a reset in the later GPL code dumps I saw, very likely to avoid this issue. However, I would not bet that this GPIO is always there, especially since it is not used in the earlier source drops, which were often quite specific to a particular device. My feeling is that the companies that did the board designs were fighting with the issue quite a bit, too.
1 Like
You're right that this requires the vendor to have included a reset GPIO in their design. I'm currently playing around with the watchdog timer, but I fear that will only provide the same (broken) reset. Like you note, the lock-up also happens after reset, so by then the watchdog will already be disabled...
1 Like
Awesome, thanks!
I also have a DGS-1210-26 and would like to run OpenWRT on it.
Yes, the DGS-1210-26 has no combo ports. All 24 copper ports and 2 SFP slots can be used simultaneously.
Did someone tried to create an OpenWRT image that can be flashed using stock firmware in order to avoid opening the case and break the warranty sticker?
I have found https://memo205.wordpress.com/2021/09/20/dgs-1210-28-f1-ファームウェアメモ/ and it appears all required information about D-Link's image format is known or included in the GPL sources.
Meanwhile I also got a DGS-1210-28 and for fun traced out its SFP I2C lines. It seems that it uses the exact same pin assignment as given by @musashino for the ApresiaLightGS120GT-SS. Only difference: There is no "LOOP" led in that device.
The reset button input and a SoC reset output (which works!) are also connected exactly the same, probably all copied from some reference design.
1 Like
What is the right way to specify the SFPs in the DTS definitions for those shared copper/fiber ports?
I tried it like this for the DGS-1210-28:
/* SFP slot, port 25 */
i2c0: i2c-gpio-0 {
compatible = "i2c-gpio";
sda-gpios = <&gpio1 6 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
scl-gpios = <&gpio1 7 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
i2c-gpio,delay-us = <2>;
#address-cells = <1>;
#size-cells = <0>;
};
sfp0: sfp-p25 {
compatible = "sff,sfp";
i2c-bus = <&i2c0>;
mod-def0-gpio = <&gpio1 8 GPIO_ACTIVE_LOW>;
los-gpio = <&gpio1 9 GPIO_ACTIVE_HIGH>;
};
Of course also enable the rtl8231:
&gpio1 {
status = "okay";
};
And then replace the
SWITCH_PORT(24, 25, qsgmii)
entry with a
port@24 {
reg = <24>;
label = "lan25";
phy-mode = "qsgmii";
sfp = <&sfp0>;
};
and this makes for example querying the eeprom/diagnostic data with ethtool -m lan25 possible.
But then I can get neither copper nor fiber links to work. I can do something with setting the port it to either mii or fibre using ethtool -s (there is also the option tp, which does nothing):
root@OpenWrt:/etc/config# ethtool -s lan25 port fibre
[11698.006650] rtl8380_rtl8214fc_media_set: port 24, set_fibre: 1
[11698.024069] Current media 3
[11698.027288] Powering off COPPER
[11698.057640] RTL8380 Link change: status: 1, ports 1000000
root@OpenWrt:/etc/config# ethtool -s lan25 port mii
[11698.088529] Powering on FIBRE
[11704.054310] rtl8380_rtl8214fc_media_set: port 24, set_fibre: 0
[11704.075065] Current media 1
[11704.078280] Powering off FIBRE
[11704.129827] Powering on COPPER
[11706.796739] RTL8380 Link change: status: 1, ports 1000000
This also enables/disable the SFP module and the copper port, their link goes on/off on the other side of the fibre or copper cables - so there is light/power on the links.
What is the right definition for the shared copper + SFP ports?
Or is this simply not yet implemented, and putting an sfp node in there confuses the programmed logic? Without specifying them using ethtool -s lan25 port fibre or mii works as intended.
You'd need a phy driver with support for the SFP port. And the port entry int the DTS should point to this phy which then points to the SFP slot. I don't know if this is implemented for this realtek phy?
I was wondering if Open vSwitch is able to use the switch hardware in these devices or if it's limited to software (i.e. the weak CPU). I'm suspecting the answer is no, it's probably too soon to discuss this when hardware support is still not 100% done, but I think it's an important thing to have eventually
My understanding is that this relies on powerful CPUs or at least OpenFlow compatible hardware. A target device could be one with an RTL931x, which supports OpenFlow and there are devices with 1GB DRAM and a decent CPU-performance. Sebastian and I have been able to squeze the first ping out of such a device a couple of days ago.
1 Like
If it doesnt exist already then it can be simply implemented using the generic SFP ops, you just want to validate the modes modules support against the PHY and then the generic OPS will attach the SFP to the upstream ethernet device.
But if you need to reconfigure the PHY into fiber mode you can do that as well upon insert, but note that this requires the SFP cage to be compatible with the generic SFP bus driver meaning it needs I2C and GPIOS-s
Afaik OpenFlow is the name of the management protocol, There are "OpenFlow" switches that provide that, but it is just an abstract API provided by the management CPU. The backend ASIC can be very different.
OpenvSwitch is an opensource software and kernel driver for Linux to do software-defined networking that supports this protocol, but it has "hardware accelerated data paths" too.
I did some digging and according to this https://hareshkhandelwal.blog/2020/03/11/lets-understand-the-openvswitch-hardware-offload/
If you enable the hardware acceleration options the first packets hit the CPU but after it has figured out where each packet stream belongs it uses "TC flower utility" to manage the switch hardware to handle the rest.
Once all flows pertaining to a particular traffic stream are formed, ovs will use TC flower utility to push and program these flows on NIC hardware.
...
Ovs uses TC data path when hw-offload flag is enabled. TC flower is an iproute2 utility and used to write data path flows on hardware.
You were talking about using tc flower to test switching hardware offload in this PR https://github.com/openwrt/openwrt/pull/4535#issuecomment-917434775 (that was merged a few hours ago, btw)
Does this mean that we should get the same hardware acceleration on OpenvSwitch too without additional work?