Support for RTL838x based managed switches

It does work now with the patch I linked above: I didn't have the (non-working) SFP ports in the dts, that's where the shift came from. After adding the remaining four ports, the LEDs are working properly.

Now that I know the register values, I try to replicate it without the patch and debugfs.

With the feedback of @svanheule I updated the curent clock driver PR. If you like you can test https://github.com/openwrt/openwrt/pull/10574 Everything should be back to normal and dmesg will show the following for devices like yours.

rtcl_unlock_registers: : registers unlocked

That is correct, those patches are part of (the to be split) https://github.com/openwrt/openwrt/pull/10453

That branch is still being slightly in flux while I work out some kinks.

image
Are you sure? Sometimes your build-cache can be tainted. Try make oldconfig or nuke your build_dir/target-mips_mips_24kc_musl builddir.

I know DAC support is hit-miss according to birger, and SFP+ modules are also very little tested. I wouldn't hold my breath on SFP+ support just yet.
On the upside, I did learn from ZyXEL that we have a 3.3Watt combined power budget for the SFP ports! So at least that is not much of a mystery anymore. Not sure if the SFP framework can deal with shared power domains though.

Routing VLAN's requires L3 support, of which not everything is in place afaik. You can still do it via the CPU of course.
Personally, I have not enough experience with DSA so I dot' know well enough how it works yet. @Borromini seems to be an expert though; so hoping I can learn a bit frm him soon :slight_smile:

You're flattering me. Took me half a day to set up a tagged WAN on my router cause I forgot I had used that VLAN internally already :upside_down_face:

FWIW my XGS1010-12 arrived today and I built dev/add_1210_1010_support (a73219f8) and booted via YMODEM (i.e. from ram rather than touching the flash). All seems to have come up fine from basic testing. I've also got a pair of 10G baseT SFP+ modules so I tried one in the switch and it was detected fine.

The oddity comes with the fact that it gets reported as a 10G only fibre device, but will happily link at lower speeds and pass traffic - I plugged in a 2.5Gb/s device to the SFP+ module and the link came up, the 2.5G end reported 2.5G speeds and the XGS1010-12 just kept reporting that it was a 10G link. Traffic was able to pass fine. I also did a 10G connection but the remote end was the same type of SFP+ module so gave no additional information.

Module is an ipolex branded ASF-10G-T which I believe has a Marvell 88X3300 multigig capable PHY inside it. ISTR there's some device tree magic that can be done to actually expose this via ethtool, so I'll try to have a play around to confirm what speeds I'm actually linking at. I'm not setup to do an iperf or similar as an alternative method of confirming link speed, but I'll see what I can do.

2 Likes

Stupid question: How do I get multicast working?

My old TV switch is dying and I thought I could just drop-in the GS108Tv3 I have. But that wasn't as easy as I imagined.. I'm unable to figure out how to get the switch to forward IGMP packets from one port to another. Snooping would also be nice, but first I have to get something through.

It's a pretty simple setup. Untagged port with decoder and tagged uplink port. Only those two ports are members of the TV VLAN. I want and expect IGMP queries and reports to be forwarded to all ports in this VLAN. But they aren't. Looks like switch is eating all incoming IGMP.

Adding a local interface in the same VLAN on the switch, I can see the IGMP packets coming in from both uplink and decoder ports. But they don't go out anywhere.

Any hints?

I have seen this issue as well. It looks like rtl838x_decode_tag doesn't read the reason field correctly. As a result l2_offloaded which is used to set the offload_fwd_mark bit on the skb is wrong. So the kernel doesn't know it has to forward the IGMP/MLD packets.

However, I later noticed that with some VLAN configuration the IGMP/MLD packets seem to be forwarded regardless. So there may be another issue, but I haven't looked into this any further.

Here is the patch I experimented with at the time: https://gist.github.com/janh/bdbf14879f685f92ec0660b4d123188f

1 Like

Thanks! Really helps to know I'm not alone :slight_smile:

I'll set up a more family-friendly test bed than in the TV loop and experiment with your patch

EDIT: Done that now, and it looks good. I now see the IGMP packets coming out of the switch. I turned on the debug site you modified in rtl838x_decode_tag() and see for example:

[  369.120000] rtl838x_decode_tag: Reason: 15 (cpu_tag 0000 04c8 0580 100f 120f 0000 0000 0000 0000 0000)
[  369.310000] rtl838x_decode_tag: Reason: 15 (cpu_tag 0000 04c8 0580 10cb 120f 0000 0000 0000 0000 0000)
[  369.390000] rtl838x_decode_tag: Reason: 5 (cpu_tag 0000 0408 0580 10cb 1205 0000 0000 0000 0000 0000)
[  369.400000] rtl838x_decode_tag: Reason: 5 (cpu_tag 0000 0408 0580 10cb 1205 0000 0000 0000 0000 0000)
[  369.480000] rtl838x_decode_tag: Reason: 15 (cpu_tag 0000 04c8 0580 1067 120f 0000 0000 0000 0000 0000)
[  369.530000] rtl838x_decode_tag: Reason: 5 (cpu_tag 0000 0408 0580 10cb 1205 0000 0000 0000 0000 0000)
[  369.550000] rtl838x_decode_tag: Reason: 5 (cpu_tag 0000 0408 0580 10cb 1205 0000 0000 0000 0000 0000)
[  369.670000] rtl838x_decode_tag: Reason: 5 (cpu_tag 0000 0408 0580 10cb 1205 0000 0000 0000 0000 0000)
[  369.680000] rtl838x_decode_tag: Reason: 5 (cpu_tag 0000 0408 0580 10cb 1205 0000 0000 0000 0000 0000)
[  369.950000] rtl838x_decode_tag: Reason: 15 (cpu_tag 0000 0408 0580 100f 120f 0000 0000 0000 0000 0000)
[  369.970000] rtl838x_decode_tag: Reason: 15 (cpu_tag 0000 0448 0580 1007 120f 0000 0000 0000 0000 0000)
[  370.310000] rtl838x_decode_tag: Reason: 15 (cpu_tag 0000 0408 0580 100f 120f 0000 0000 0000 0000 0000)
[  370.740000] rtl838x_decode_tag: Reason: 6 (cpu_tag 0000 04e8 0580 1067 1206 0000 0000 0000 0000 0000)
[  370.920000] rtl838x_decode_tag: Reason: 5 (cpu_tag 0000 0408 0580 10cb 1205 0000 0000 0000 0000 0000)

where the "Reason: 6" matches an IGMP packet as expected from the patch.

IIUC your patch fixes two separate issues: off-by-one in the reason/queue decoding of the tag and disabling L2 offload of "NIC_RX_REASON_RMA" packets.

But it looks like only the first part is actually required for IGMP, is that correct? They are tagged as "NIC_RX_REASON_SPECIAL_TRAP", and should have been caught by the old code as well if the reason decode had been correct.

EDIT2: looking a bit more on that off-by-one thing. From sdk/system/include/drv/nic/r8380.h I see that the RX tag is supposed to look like this:

        struct {
            uint8   CPUTAGIF;
            
            uint8   QID:3;
            uint8   SPN:5;
            
            uint16  MIR_HIT:4;
            uint16  ACL_HIT:1;
            uint16  ACL_IDX:11;
            
            uint16          :2;
            uint16  OTAGIF:1;
            uint16  ITAGIF:1;
            uint16  RVID:12;

            uint8          :1;
            uint8   MAC_CST:1;
            uint8   ATK_HIT:1;
            uint8   ATK_TYPE:5;
            
            uint8   NEW_SA:1;
            uint8   L2_PMV:1;
            uint8           :2;
            uint8   REASON:4;
           
            uint8   RSV0;
            uint8   RSV1;
        } __attribute__ ((aligned(1), packed)) rx;

which does put the REASON in the 3rd 16bit word like the present code suggests. But the tag examples above shows that our cpu_tag array is already off-by-one compared to this struct. I believe the CPUTAGIF is supposed to by constant 4, which is identifid as "PROTO ID" on TX by the SDK. This puts the start of the tag struct at h->cpu_tag[1] and not h->cpu_tag[0].

We can also confirm this by looking at the RVID field. The IGMP packet is on VLAN 103 (0x67). The "my mac" (reason 5) packets are on VLAN 203 (0xcb). This field is obiously h->cpu_tag[3].

So the off-by-one is definitely there, and is the primary reason the IGMP packets fail.

But the original reason macthing is also off:

      if (t->reason != 4) // NIC_RX_REASON_SPECIAL_TRAP
               t->l2_offloaded = 1;

Which offloads everything but NIC_RX_REASON_INNER_OUTTER_CFI. This does not make any sense at all. I'm guessing someone happened to test IGMP on on a VLAN with VID & 0xf == 4, and this made it work...

The 838x reason table clearly shows that NIC_RX_REASON_SPECIAL_TRAP is 6, like your patch fixes it to:

static uint32           reasonTbl[][2] = 
{
        {0,                                                                                     0, },
        {NIC_RX_REASON_RLDP_RLPP,                                               0, },
        {NIC_RX_REASON_RMA,                                                             0, },
        {NIC_RX_REASON_IGR_VLAN_FILTER,                                 0, },
        {NIC_RX_REASON_INNER_OUTTER_CFI,                                0, },
        {NIC_RX_REASON_MY_MAC,                                                  0, },
        {NIC_RX_REASON_SPECIAL_TRAP,                                    0, },
        {NIC_RX_REASON_SPECIAL_COPY,                                    0, },
        {NIC_RX_REASON_ROUTING_EXCEPTION,                               0, },
        {NIC_RX_REASON_UNKWN_UCST_MCST,                                 0, },
        {NIC_RX_REASON_MAC_CONSTRAINT_SYS,                              NIC_RX_REASON_MAC_CONSTRAINT, },
        {NIC_RX_REASON_MAC_CONSTRAINT_VLAN,                             NIC_RX_REASON_MAC_CONSTRAINT, },
        {NIC_RX_REASON_MAC_CONSTRAINT_PORT,                             NIC_RX_REASON_MAC_CONSTRAINT, },
        {NIC_RX_REASON_CRC_ERROR,                                               0, },
        {NIC_RX_REASON_IP6_UNKWN_EXT_HDR,                               0, },
        {NIC_RX_REASON_NORMAL_FWD,                                              0, },
};

EDIT3: I'lll stop up, I promise. But I just realised how this affects us with non-IGMP. It means that we clear l2_offloaded on every single packet received on VLANs 4, 20, 36, 52, etc... How does that affect performance if you use one of those VLANs?

3 Likes

New post since I've stopped editing :slight_smile:

I see now where we got the offset. The SDK defines the 838x packet header as:

typedef struct nic_pkthdr_s
{
    /* word [0] */ 
    uint8 *buf_addr;
#ifdef __LITTLE_ENDIAN
    /* word [1] */
    uint16  buf_size:14;
    uint16          :2;
    uint16  reserved;

    /* word [2] */    
    uint16  buf_len:14;
    uint16          :2;
    uint16  pkt_offset:14;
    uint16      :1;
    uint16  more:1;

//#error fix the CPU tag format to support big litten
    /* word [3] */    
    uint16  Reserved_1; /*cw it may cause problem here, fix this*/

#else /* if big endian*/

    /* word [1] */
    uint16  reserved;
    uint16          :2;
    uint16  buf_size:14;

    /* word [2] */    
    uint16  more:1;
    uint16      :1;
    uint16  pkt_offset:14;
    uint16          :2;
    uint16  buf_len:14;

    /* word [3] */    
    uint16  Reserved_1;
#endif
    /* word [3] ~ word [5]*/       
    maple_nic_cpuTag_t    cpuTag;

    /* Used by Software */
    struct drv_nic_pkt_s *packet;
    uint32  *ring_entry;
    drv_nic_tx_cb_f tx_callback;
    void    *cookie;
} nic_pkthdr_t;

This is simplified to

struct p_hdr {
	uint8_t		*buf;
	uint16_t	reserved;
	uint16_t	size;		/* buffer size */
	uint16_t	offset;
	uint16_t	len;		/* pkt len */
	uint16_t	cpu_tag[10];
} __packed __aligned(1);

in our driver. The missing part is the uint16 Reserved_1 right before the cpu tag. So that's what we map into cpu_tag[0], causing the real tag to start at cpu_tag[1].

And this seems to have been an intentional change in this giga-patch: https://github.com/openwrt/openwrt/commit/dc9cc0d3e2a1e

I assume it was intentional since the patch also adds this peculiar reserved cpu_rag[0] handling to the TX path. So it was probably supposed to shift on RX too?

<rant>
But I absolutely hate having to guess like this. If this had been properly broken up and documented, then it would have been possible to review it back in January 2021. And if the bug still slipped though then we would at least known why it was added.

Like it is now, I am staring at a commit claming to "add QoS and rate control", while removing a "reserved" field from a struct supposed to match a hardware defined header. That makes no sense at all. Taking a shortcut, or just buggy?
</rant>

1 Like

Wait, what? Huge code dumps getting merged? :stuck_out_tongue:

Nice to see that you've found all these little errors! Looking forward to the fixup patches :wink:
I think this is a good argument for giving new code at least more than a cursory glance, and not just merge piles of patches for the sake of the "progress" that the commit messages claim to make.

The CPU tags are also the register docs website, and can be reached by following the "CPU tags" links in the platform boxes. See for example https://svanheule.net/realtek/maple/cputag. The fields are listed by bit offset, so there shouldn't be any confusion w.r.t. endianness.

Heh, yes :slight_smile: I sent the minimal fixup now, after a brief real TV test. Works for me

3 Likes

hi is it possible to run openwrt on tp-link T1700G-28TQ ?

According to the GPL-dump, v1 and v2 are Broadcom-based, so: no. Do you have a newer version than v2?

yes i have v2 and for this one firmware from v3 works too , thanks for reply

Ah, I guess this explains why I saw different behaviour depending on the VLAN configuration without the patch, as I am actually using VLAN 20 here…

Thank you for investigating and submitting the patch!

Only thing I noticed is that the patch also adds a comment about the reserved field to rtl839x_decode_tag, but keeps the offsets for queue and crc_error. According to the documentation, those are also wrong (and crc_error should actually use BIT(6)).

2 Likes

Thanks. You're right. I missed that. Too many magic numbers and offsets here, and I'm lazy. So I stopped after checking half-way through the 839x function and finding that previous fixup commit, assuming it was complete.

I'll send a followup-fix.

2 Likes

Fun side effect of my TV switch replacement is the reduced power consumption. I exchanged a Cisco/Linksys SLM2008 bough 2009 with a Netgear GS108Tv3 bought 2020. The feature set is identical: 8 gig-ports with optional PoE input on port 1. The SLM2008 ran the latest OEM firmware. The GS108Tv3 is running current OpenWrt master.

This graph shows the consumption as measured by the upstream PSE switch. Traffic load and connected ports are the same:
index

2 Likes

I'm still trying to figure out if we can come up with some sort of 'test-bed' where we can tests all this exotic features ... This way, we can check if the features are functional as expected; and if we get regressions. But I'm too far of a network expect for this and don't have enough hardware for a test-bench (yet?)...

Found the issue, now it's fine.

However, routing vlans doesn't work at all, not even by the CPU. The only way I got routing to work is by removing ports from the switch bridge using them as dedicated networks. The speed I got was around 155Mbp/s.

Another issue is the 2.5G ports failing randomly.

Log

[ 492.032175] rtl930x_switch_irq link faults: 0cffff00
[ 492.037702] rtl930x_switch_irq RX symbol errors: 000000c4
[ 492.489556] rtl93xx_phylink_mac_config port 24, mode 0, phy-mode: hsgmii, speed 2500, link 1
[ 492.498992] rtl93xx_phylink_mac_config SDS is 6
[ 492.505070] rtl93xx_phylink_mac_config CURRENT FORCED MODE 18
[ 492.511456] rtl9300_rtl8226_mode_set setting serdes 6 to mode sgmii +++++
[ 492.519055] rtl930x_switch_irq link faults: 0cffff00
[ 492.524569] rtl930x_switch_irq RX symbol errors: 000000c4
[ 492.550577] rtl9300_force_sds_mode: SDS: 6, PHY mode 0
[ 492.556326] rtl9300_force_sds_mode: forcing SDS mode 1f
[ 492.570159] rtl9300_serdes_mac_link_config: registers before 00000000 00001403
[ 492.582218] rtl9300_serdes_mac_link_config: registers after 00000000 00001403
[ 492.601151] rtl930x_switch_irq link faults: 0dffff00
[ 492.606670] rtl930x_switch_irq RX symbol errors: 000000c4
[ 492.688678] rtl9300_force_sds_mode: SDS: 6, PHY mode 4
[ 492.694426] rtl9300_force_sds_mode: forcing SDS mode 2
[ 492.921806] rtl9300_force_sds_mode toggling LC or Ring for 10gr, round 0
[ 492.956265] rtl9300_force_sds_mode end power 0x20 0 30
[ 492.961978] rtl9300_force_sds_mode -------------------- serdes 6 forced to 2 DONE
[ 492.970342] rtl9300_sds_tx_config SerDes 6, pre-amp enable 0, pre-amp val 0, main-amp 9, post-amp enable 1, post-amp val 8, impedance 8
[ 492.995953] rtl83xx-switch switch@1b000000 lan9: Link is Up - 2.5Gbps/Full - flow control off
[ 493.009355] rtl83xx-switch switch@1b000000 lan9: Link is Down
[ 493.019944] rtl93xx_phylink_mac_config port 24, mode 0, phy-mode: hsgmii, speed 2500, link 1
[ 493.029414] rtl93xx_phylink_mac_config SDS is 6
[ 493.035506] rtl93xx_phylink_mac_config CURRENT FORCED MODE 2
[ 493.041796] rtl9300_rtl8226_mode_set setting serdes 6 to mode hsgmii +++++
[ 493.049494] rtl930x_switch_irq link faults: 0dffff00
[ 493.055006] rtl930x_switch_irq RX symbol errors: 000000c4
[ 493.081013] rtl9300_force_sds_mode: SDS: 6, PHY mode 0
[ 493.086774] rtl9300_force_sds_mode: forcing SDS mode 1f
[ 493.100611] rtl9300_serdes_mac_link_config: registers before 00000000 00001403
[ 493.112665] rtl9300_serdes_mac_link_config: registers after 00000000 00001403
[ 493.131151] rtl930x_switch_irq link faults: 0dffff00
[ 493.136668] rtl930x_switch_irq RX symbol errors: 000000c4
[ 493.219120] rtl9300_force_sds_mode: SDS: 6, PHY mode 17
[ 493.224969] rtl9300_force_sds_mode: forcing SDS mode 12
[ 493.452426] rtl9300_force_sds_mode toggling LC or Ring for 10gr, round 0
[ 493.486882] rtl9300_force_sds_mode end power 0x20 0 30
[ 493.492592] rtl9300_force_sds_mode -------------------- serdes 6 forced to 12 DONE
[ 493.501045] rtl9300_sds_tx_config SerDes 6, pre-amp enable 0, pre-amp val 0, main-amp 9, post-amp enable 1, post-amp val 8, impedance 8
[ 493.526665] rtl83xx-switch switch@1b000000 lan9: Link is Up - 2.5Gbps/Full - flow control off
[ 493.561885] rtl83xx-switch switch@1b000000 lan9: Link is Down
[ 493.587329] rtl93xx_phylink_mac_config port 24, mode 0, phy-mode: hsgmii, speed 2500, link 1
[ 493.596777] rtl93xx_phylink_mac_config SDS is 6
[ 493.602810] rtl93xx_phylink_mac_config CURRENT FORCED MODE 18
[ 493.609249] rtl9300_rtl8226_mode_set setting serdes 6 to mode sgmii +++++
[ 493.616858] rtl930x_switch_irq link faults: 0dffff00
[ 493.622379] rtl930x_switch_irq RX symbol errors: 000000c4
[ 493.648395] rtl9300_force_sds_mode: SDS: 6, PHY mode 0
[ 493.654108] rtl9300_force_sds_mode: forcing SDS mode 1f
[ 493.667954] rtl9300_serdes_mac_link_config: registers before 00000000 00001403
[ 493.680009] rtl9300_serdes_mac_link_config: registers after 00000000 00001403
[ 493.698151] rtl930x_switch_irq link faults: 0dffff00
[ 493.703664] rtl930x_switch_irq RX symbol errors: 000000c4
[ 493.786465] rtl9300_force_sds_mode: SDS: 6, PHY mode 4
[ 493.792172] rtl9300_force_sds_mode: forcing SDS mode 2
[ 494.019579] rtl9300_force_sds_mode toggling LC or Ring for 10gr, round 0
[ 494.054037] rtl9300_force_sds_mode end power 0x20 0 30
[ 494.059784] rtl9300_force_sds_mode -------------------- serdes 6 forced to 2 DONE
[ 494.068153] rtl9300_sds_tx_config SerDes 6, pre-amp enable 0, pre-amp val 0, main-amp 9, post-amp enable 1, post-amp val 8, impedance 8
[ 494.093748] rtl83xx-switch switch@1b000000 lan9: Link is Up - 2.5Gbps/Full - flow control off
[ 494.107201] rtl83xx-switch switch@1b000000 lan9: Link is Down
[ 494.118025] rtl93xx_phylink_mac_config port 24, mode 0, phy-mode: hsgmii, speed 2500, link 1
[ 494.127470] rtl93xx_phylink_mac_config SDS is 6
[ 494.133503] rtl93xx_phylink_mac_config CURRENT FORCED MODE 2
[ 494.139844] rtl9300_rtl8226_mode_set setting serdes 6 to mode hsgmii +++++
[ 494.147549] rtl930x_switch_irq link faults: 0dffff00
[ 494.153061] rtl930x_switch_irq RX symbol errors: 000000c4
[ 494.179077] rtl9300_force_sds_mode: SDS: 6, PHY mode 0
[ 494.184830] rtl9300_force_sds_mode: forcing SDS mode 1f
[ 494.198661] rtl9300_serdes_mac_link_config: registers before 00000000 00001403
[ 494.210720] rtl9300_serdes_mac_link_config: registers after 00000000 00001403
[ 494.229151] rtl930x_switch_irq link faults: 0dffff00
[ 494.234672] rtl930x_switch_irq RX symbol errors: 000000c4
[ 494.317191] rtl9300_force_sds_mode: SDS: 6, PHY mode 17
[ 494.322996] rtl9300_force_sds_mode: forcing SDS mode 12
[ 494.550493] rtl9300_force_sds_mode toggling LC or Ring for 10gr, round 0
[ 494.584968] rtl9300_force_sds_mode end power 0x20 0 30
[ 494.590682] rtl9300_force_sds_mode -------------------- serdes 6 forced to 12 DONE
[ 494.599136] rtl9300_sds_tx_config SerDes 6, pre-amp enable 0, pre-amp val 0, main-amp 9, post-amp enable 1, post-amp val 8, impedance 8
[ 494.624756] rtl83xx-switch switch@1b000000 lan9: Link is Up - 2.5Gbps/Full - flow control off
[ 494.638105] rtl83xx-switch switch@1b000000 lan9: Link is Down
[ 494.685820] rtl93xx_phylink_mac_config port 24, mode 0, phy-mode: hsgmii, speed 2500, link 1
[ 494.695251] rtl93xx_phylink_mac_config SDS is 6
[ 494.701279] rtl93xx_phylink_mac_config CURRENT FORCED MODE 18
[ 494.707718] rtl9300_rtl8226_mode_set setting serdes 6 to mode sgmii +++++
[ 494.715325] rtl930x_switch_irq link faults: 0dffff00
[ 494.720838] rtl930x_switch_irq RX symbol errors: 000000c4
[ 494.746851] rtl9300_force_sds_mode: SDS: 6, PHY mode 0
[ 494.752559] rtl9300_force_sds_mode: forcing SDS mode 1f
[ 494.766411] rtl9300_serdes_mac_link_config: registers before 00000000 00001403
[ 494.778470] rtl9300_serdes_mac_link_config: registers after 00000000 00001403
[ 494.797151] rtl930x_switch_irq link faults: 0dffff00
[ 494.802668] rtl930x_switch_irq RX symbol errors: 000000c4
[ 494.884936] rtl9300_force_sds_mode: SDS: 6, PHY mode 4
[ 494.890650] rtl9300_force_sds_mode: forcing SDS mode 2
[ 495.118074] rtl9300_force_sds_mode toggling LC or Ring for 10gr, round 0
[ 495.152529] rtl9300_force_sds_mode end power 0x20 0 30
[ 495.158277] rtl9300_force_sds_mode -------------------- serdes 6 forced to 2 DONE
[ 495.166640] rtl9300_sds_tx_config SerDes 6, pre-amp enable 0, pre-amp val 0, main-amp 9, post-amp enable 1, post-amp val 8, impedance 8
[ 495.192234] rtl83xx-switch switch@1b000000 lan9: Link is Up - 2.5Gbps/Full - flow control off
[ 495.205724] rtl83xx-switch switch@1b000000 lan9: Link is Down
[ 495.213321] rtl93xx_phylink_mac_config port 24, mode 0, phy-mode: hsgmii, speed 2500, link 1
[ 495.222787] rtl93xx_phylink_mac_config SDS is 6
[ 495.228856] rtl93xx_phylink_mac_config CURRENT FORCED MODE 2
[ 495.235186] rtl9300_rtl8226_mode_set setting serdes 6 to mode hsgmii +++++
[ 495.242840] rtl930x_switch_irq link faults: 0dffff00
[ 495.248353] rtl930x_switch_irq RX symbol errors: 000000c4
[ 495.274358] rtl9300_force_sds_mode: SDS: 6, PHY mode 0
[ 495.280065] rtl9300_force_sds_mode: forcing SDS mode 1f
[ 495.293891] rtl9300_serdes_mac_link_config: registers before 00000000 00001403
[ 495.305971] rtl9300_serdes_mac_link_config: registers after 00000000 00001403
[ 495.324155] rtl930x_switch_irq link faults: 0dffff00
[ 495.329673] rtl930x_switch_irq RX symbol errors: 000000c4
[ 495.412419] rtl9300_force_sds_mode: SDS: 6, PHY mode 17
[ 495.418262] rtl9300_force_sds_mode: forcing SDS mode 12
[ 495.645755] rtl9300_force_sds_mode toggling LC or Ring for 10gr, round 0
[ 495.680216] rtl9300_force_sds_mode end power 0x20 0 30
[ 495.685970] rtl9300_force_sds_mode -------------------- serdes 6 forced to 12 DONE
[ 495.694428] rtl9300_sds_tx_config SerDes 6, pre-amp enable 0, pre-amp val 0, main-amp 9, post-amp enable 1, post-amp val 8, impedance 8
[ 495.720031] rtl83xx-switch switch@1b000000 lan9: Link is Up - 2.5Gbps/Full - flow control off
[ 495.733427] rtl83xx-switch switch@1b000000 lan9: Link is Down
[ 495.767941] rtl93xx_phylink_mac_config port 24, mode 0, phy-mode: hsgmii, speed 2500, link 1
[ 495.777387] rtl93xx_phylink_mac_config SDS is 6
[ 495.783413] rtl93xx_phylink_mac_config CURRENT FORCED MODE 18
[ 495.789851] rtl9300_rtl8226_mode_set setting serdes 6 to mode sgmii +++++
[ 495.797460] rtl930x_switch_irq link faults: 0dffff00
[ 495.802980] rtl930x_switch_irq RX symbol errors: 000000c4
[ 495.828983] rtl9300_force_sds_mode: SDS: 6, PHY mode 0
[ 495.834729] rtl9300_force_sds_mode: forcing SDS mode 1f
[ 495.848563] rtl9300_serdes_mac_link_config: registers before 00000000 00001403
[ 495.860621] rtl9300_serdes_mac_link_config: registers after 00000000 00001403
[ 495.879152] rtl930x_switch_irq link faults: 0dffff00
[ 495.884670] rtl930x_switch_irq RX symbol errors: 000000c4
[ 495.967088] rtl9300_force_sds_mode: SDS: 6, PHY mode 4
[ 495.972801] rtl9300_force_sds_mode: forcing SDS mode 2
[ 496.200194] rtl9300_force_sds_mode toggling LC or Ring for 10gr, round 0
[ 496.234675] rtl9300_force_sds_mode end power 0x20 0 30
[ 496.240391] rtl9300_force_sds_mode -------------------- serdes 6 forced to 2 DONE
[ 496.248751] rtl9300_sds_tx_config SerDes 6, pre-amp enable 0, pre-amp val 0, main-amp 9, post-amp enable 1, post-amp val 8, impedance 8
[ 496.274355] rtl83xx-switch switch@1b000000 lan9: Link is Up - 2.5Gbps/Full - flow control off
[ 496.301568] rtl83xx-switch switch@1b000000 lan9: Link is Down
[ 496.309212] rtl93xx_phylink_mac_config port 24, mode 0, phy-mode: hsgmii, speed 2500, link 1
[ 496.318683] rtl93xx_phylink_mac_config SDS is 6
[ 496.324761] rtl93xx_phylink_mac_config CURRENT FORCED MODE 2
[ 496.331050] rtl9300_rtl8226_mode_set setting serdes 6 to mode hsgmii +++++
[ 496.338744] rtl930x_switch_irq link faults: 0dffff00
[ 496.344258] rtl930x_switch_irq RX symbol errors: 000000c4
[ 496.370257] rtl9300_force_sds_mode: SDS: 6, PHY mode 0
[ 496.376006] rtl9300_force_sds_mode: forcing SDS mode 1f
[ 496.389840] rtl9300_serdes_mac_link_config: registers before 00000000 00001403
[ 496.401900] rtl9300_serdes_mac_link_config: registers after 00000000 00001403
[ 496.420152] rtl930x_switch_irq link faults: 0dffff00
[ 496.425667] rtl930x_switch_irq RX symbol errors: 000000c4
[ 496.452869] rtl930x_switch_irq link faults: 0dffff00
[ 496.458387] rtl930x_switch_irq RX symbol errors: 000000c4
[ 496.519903] rtl9300_force_sds_mode: SDS: 6, PHY mode 17
[ 496.525752] rtl9300_force_sds_mode: forcing SDS mode 12
[ 496.753221] rtl9300_force_sds_mode toggling LC or Ring for 10gr, round 0
[ 496.787678] rtl9300_force_sds_mode end power 0x20 0 30
[ 496.793408] rtl9300_force_sds_mode -------------------- serdes 6 forced to 12 DONE
[ 496.801881] rtl9300_sds_tx_config SerDes 6, pre-amp enable 0, pre-amp val 0, main-amp 9, post-amp enable 1, post-amp val 8, impedance 8
[ 496.827488] rtl83xx-switch switch@1b000000 lan9: Link is Up - 2.5Gbps/Full - flow control off
[ 496.838416] rtl930x_switch_irq link faults: 0dffff00
[ 496.843932] rtl930x_switch_irq RX symbol errors: 000000c4
[ 496.852686] rtl930x_switch_irq link faults: 0cffff00
[ 496.858215] rtl930x_switch_irq RX symbol errors: 000000c4
[ 496.864302] rtl83xx-switch switch@1b000000 lan9: Link is Down