I think I'm onto something.. I've modified the qca8k driver to include support for connecting PHY4 to CPU port 6 through RGMII. I can shortly see PHY4 is up and observe some activity on the internal IPQ5018 gephy:
[ 4.807908] regi_init[2548]:INFO:Initializing SCOMPHY Done!!
[ 4.808107] regi_init[2574]:INFO:qca-ssdk module init succeeded!
[ 4.815724] dp1: ppe offload disabled: 0 for macid 1
[ 4.818724] dp1: Switch attached to macid 1 status: 0
[ 4.823685] nss-dp 39c00000.dp1 (unnamed net_device) (uninitialized): nss_dp_gmac: Registering netdev eth%d(qcom-id:1) with GMAC, mac_base: 0xffffffc081fe0000
[ 4.897273] Qualcomm IPQ5018 internal PHY 88000.mdio-1:07: attached PHY driver (mii_bus:phy_addr=88000.mdio-1:07, irq=POLL)
[ 4.899730] dp2: ppe offload disabled: 0 for macid 2
[ 4.907353] dp2: Switch attached to macid 2 status: 0
[ 4.912587] nss-dp 39d00000.dp2 wan (uninitialized): nss_dp_gmac: Registering netdev wan(qcom-id:2) with GMAC, mac_base: 0xffffffc082310000
[ 5.010659] Qualcomm QCA8081 90000.mdio-1:1c: attached PHY driver (mii_bus:phy_addr=90000.mdio-1:1c, irq=POLL)
[ 5.012922] **********************************************************
[ 5.019661] * NSS Data Plane driver
[ 5.026087] **********************************************************
[ 5.046473] kmodloader: done loading kernel modules from /etc/modules-boot.d/*
[ 5.051110] init: - preinit -
[ 5.087112] qca8k 90000.mdio-1:11: PHY [90000.mdio-1:04] driver [Qualcomm Atheros 8337 internal PHY] (irq=POLL)
[ 5.089728] qca8k 90000.mdio-1:11: configuring for phy/rgmii link mode
[ 5.098670] qca8k 90000.mdio-1:11: PHY4 connected to CPU port 6 through RGMII interface
[ 5.198087] qca8k 90000.mdio-1:11 lan1 (uninitialized): PHY [90000.mdio-1:01] driver [Qualcomm Atheros 8337 internal PHY] (irq=POLL)
[ 5.271940] mtdblock: MTD device 'devinfo' is NAND, please consider using UBI block devices instead.
[ 5.277660] qca8k 90000.mdio-1:11 lan2 (uninitialized): PHY [90000.mdio-1:02] driver [Qualcomm Atheros 8337 internal PHY] (irq=POLL)
[ 5.357824] qca8k 90000.mdio-1:11 lan3 (uninitialized): PHY [90000.mdio-1:03] driver [Qualcomm Atheros 8337 internal PHY] (irq=POLL)
[ 5.366501] nss-dp 39c00000.dp1 eth0: entered promiscuous mode
[ 5.369051] DSA: tree 0 setup
[ 6.346849] random: crng init done
[ 6.517336] qca8k 90000.mdio-1:11 lan1: configuring for phy/gmii link mode
....
[ 9.609217] qca8k 90000.mdio-1:11 lan1: Link is Up - 1Gbps/Full - flow control rx/tx
....
[ 10.783049] qca8k 90000.mdio-1:11 lan1: Link is Down
[ 19.687674] qca8k 90000.mdio-1:11: Link is Up - 100Mbps/Full - flow control off
[ 20.007277] Qualcomm IPQ5018 internal PHY 88000.mdio-1:07: Downshift occurred from negotiated speed 1Gbps to actual speed 100Mbps, check cabling!
[ 20.007406] nss-dp 39c00000.dp1 eth0: PHY Link up speed: 100
root@OpenWrt:~# mdio 9*
DEV PHY-ID LINK
0x00 0x004dd036 down
0x01 0x004dd036 down
0x02 0x004dd036 down
0x03 0x004dd036 down
0x04 0x004dd036 up
0x10 0x00000000 down
0x11 0x00000000 down
0x12 0x0c720000 down
0x13 0x00000000 down
0x14 0x00000000 down
0x15 0x00000000 up
0x16 0x00000000 down
0x17 0x00000000 down
0x18 0x00000000 down
0x1c 0x004dd101 down
0x1d 0x8aa21a08 down
[ 23.458911] nss-dp 39c00000.dp1 eth0: PHY Link is down
[ 23.491456] qca8k 90000.mdio-1:11 lan1: configuring for phy/gmii link mode
[ 23.542847] br-lan: port 1(lan1) entered blocking state
[ 23.542924] br-lan: port 1(lan1) entered disabled state
[ 23.552438] qca8k 90000.mdio-1:11 lan1: entered allmulticast mode
[ 23.552508] nss-dp 39c00000.dp1 eth0: entered allmulticast mode
[ 23.567762] qca8k 90000.mdio-1:11 lan1: entered promiscuous mode
[ 23.640335] qca8k 90000.mdio-1:11 lan2: configuring for phy/gmii link mode
[ 23.641994] br-lan: port 2(lan2) entered blocking state
[ 23.646148] br-lan: port 2(lan2) entered disabled state
[ 23.667245] qca8k 90000.mdio-1:11 lan2: entered allmulticast mode
[ 23.670174] qca8k 90000.mdio-1:11 lan2: entered promiscuous mode
[ 23.707408] qca8k 90000.mdio-1:11 lan3: configuring for phy/gmii link mode
[ 23.708704] br-lan: port 3(lan3) entered blocking state
[ 23.713210] br-lan: port 3(lan3) entered disabled state
[ 23.721537] qca8k 90000.mdio-1:11 lan3: entered allmulticast mode
[ 23.729803] qca8k 90000.mdio-1:11 lan3: entered promiscuous mode
[ 23.849732] qca8k 90000.mdio-1:11: Link is Down
[ 26.647211] qca8k 90000.mdio-1:11 lan1: Link is Up - 1Gbps/Full - flow control rx/tx
[ 26.647325] adpt_mp_port_netdev_change_notify[1186]:ERROR:netdev change notify with incorrect port 0
[ 26.654056] ssdk_dev_event[2313]:ERROR:netdev change notify failed
[ 26.663289] br-lan: port 1(lan1) entered blocking state
[ 26.669215] br-lan: port 1(lan1) entered forwarding state
root@OpenWrt:~# mdio 9*
DEV PHY-ID LINK
0x00 0x004dd036 down
0x01 0x004dd036 up
0x02 0x004dd036 down
0x03 0x004dd036 down
0x04 0x004dd036 down
0x10 0x00000000 down
0x11 0x55555555 up
0x12 0x00540000 down
0x13 0x00000000 down
0x14 0x00000000 down
0x15 0x00000000 down
0x16 0x00000000 down
0x17 0x00000000 down
0x18 0x00000000 down
0x1c 0x004dd101 down
0x1d 0x8aa21a08 down
so it first successfully detects the connection between the internal gephy and phy4 on the external qca8337 switch, then it looks like it (resets?) disconnects the RGMII link (at mdio address 0x04) and initializes an SGMII link (at mdio address 0x11)? Will dig deeper..
forgive me for asking a (possibly) daft question, I take it the spn56 is interfaced differently than the mx5500 which also uses the 8337 in that this use 8337 & 8081 which is unique and needs additional legwork patching the 2nd interface ?
I'm also assuming PORTX_HEADER_CTRL.is enabled for the other interface for this to be working for all ports or does it need to be for each one ?
I'm trying to understand what happens in the process of setting up the interfaces defined in /etc/config/network (as per 02_network provided). I've found out that when not defining any interfaces, the 8337 switch is actually configured properly, connecting phy4 to CPU port 6 which I set in the qca8k driver as per datasheet:
Before any interfaces are set up, the internal gephy and phy4 on the external switch show as 'up':
before
root@OpenWrt:~# mdio 8*
DEV PHY-ID LINK
....
0x07 0x004dd0c0 up <-- IPQ5018 Internal GE PHY
root@OpenWrt:~# mdio 9*
DEV PHY-ID LINK
0x00 0x004dd036 down
0x01 0x004dd036 up
0x02 0x004dd036 down
0x03 0x004dd036 down
0x04 0x004dd036 up <-- QCA8337 PHY4 (connected to CPU Port 6)
0x10 0x00000000 down
0x11 0x00000000 down
0x12 0x1df20000 down
0x13 0x00000000 down
0x14 0x00000000 down
0x15 0x00000000 up
root@OpenWrt:~# mdio 9* phy 4 raw 0x11
0x7c5c <-- BIT(10) for link status is set to 1 (UP)
root@OpenWrt:~# mdio 8* phy 7 raw 0x11
0x7cbc <-- BIT(10) for link status is set to 1 (UP)
After setting up devices and interfaces manually in /etc/config/network and trigger a /etc/init.d/network restart, both links go down:
after
root@OpenWrt:~# mdio 8*
DEV PHY-ID LINK
....
0x07 0x004dd0c0 down
root@OpenWrt:~# mdio 9*
DEV PHY-ID LINK
0x00 0x004dd036 down
0x01 0x004dd036 down
0x02 0x004dd036 up
0x03 0x004dd036 down
0x04 0x004dd036 down
0x10 0x00000000 down
0x11 0x55555555 up
0x12 0x00540014 down
0x13 0x00000000 down
0x14 0x00000000 down
0x15 0x00000000 down
dmesg: see first 5 lines and time, when it's setting up devices/interfaces. What's also interesting is that it seems to set up I think an SGMII connection and switches MAC6 (CPU port) from PHY4 in RGMII to it's SerDes interface (look at phy 11 before and after) while it shouldn't. What is triggering this?
dmesg
[ 19.687242] qca8k 90000.mdio-1:11: Link is Up - 100Mbps/Full - flow control off
[ 20.016982] Qualcomm IPQ5018 internal PHY 88000.mdio-1:07: Downshift occurred from negotiated speed 1Gbps to actual speed 100Mbps, check cabling!
[ 20.017085] nss-dp 39c00000.dp1 eth0: PHY Link up speed: 100
[ 28.920836] nss-dp 39c00000.dp1 eth0: PHY Link is down
[ 28.964806] qca8k 90000.mdio-1:11 lan1: configuring for phy/gmii link mode
[ 29.011789] br-lan: port 1(lan1) entered blocking state
[ 29.011865] br-lan: port 1(lan1) entered disabled state
[ 29.017421] qca8k 90000.mdio-1:11 lan1: entered allmulticast mode
[ 29.021080] nss-dp 39c00000.dp1 eth0: entered allmulticast mode
[ 29.046352] qca8k 90000.mdio-1:11 lan1: entered promiscuous mode
[ 29.059590] qca8k 90000.mdio-1:11: Link is Down
[ 29.116698] qca8k 90000.mdio-1:11 lan2: configuring for phy/gmii link mode
[ 29.135624] br-lan: port 2(lan2) entered blocking state
[ 29.135709] br-lan: port 2(lan2) entered disabled state
[ 29.146651] qca8k 90000.mdio-1:11 lan2: entered allmulticast mode
[ 29.150615] qca8k 90000.mdio-1:11 lan2: entered promiscuous mode
[ 29.197583] qca8k 90000.mdio-1:11 lan3: configuring for phy/gmii link mode
[ 29.198964] br-lan: port 3(lan3) entered blocking state
[ 29.203393] br-lan: port 3(lan3) entered disabled state
[ 29.216961] qca8k 90000.mdio-1:11 lan3: entered allmulticast mode
[ 29.227209] qca8k 90000.mdio-1:11 lan3: entered promiscuous mode
[ 29.260734] br-lan: port 4(wan) entered blocking state
[ 29.260810] br-lan: port 4(wan) entered disabled state
[ 29.265049] nss-dp 39d00000.dp2 wan: entered allmulticast mode
[ 29.278267] nss-dp 39d00000.dp2 wan: entered promiscuous mode
[ 32.249142] qca8k 90000.mdio-1:11 lan2: Link is Up - 1Gbps/Full - flow control rx/tx
[ 32.269358] br-lan: port 2(lan2) entered blocking state
[ 32.275242] br-lan: port 2(lan2) entered forwarding state
Yes, the mx2000 and mx5500 both are connected to the qca8337 switch over a fixed link between the SOCâs MAC/uniphy and the switchâs SerDes/MAC (PORT6).
The SPNMX56 uses its uniphy to connect to the 2.5Gbps qca8081 while using its internal ethernet phy to connect to phy4 on the 8337 switch over MDI. The switchâs CPU port (port 6) can be configured to connect to phy4 over RGMII instead of SerDes, which Iâve written a patch for as the current qca8k driver doesnât support that now. But this is still a WiP..
When you add an interface to the 02_network for the device, all it does is populate the interfaces as WAN or to br-lan and bring them up as they will be down by default, it doesn't really change the low level kernel config in any way.
Are you modeling the PHY in the DTS with its PHY mode set to RGMII?
That last property is part of a patch I wrote for connecting phy4 to MAC6 (cpu port) via rgmii.
I write to a register as specified in the qca8337 datasheet. It comes up as shown above (mdio commands), but the phys on both ends go down after the interfaces are brought up.
Itâs basically a phy to phy connection over mdi.
Hm, are you sure that ethernet property is needed?
Asus RT-AX89X uses something similar where it uses PHY to PHY connection between the MAC and switch MAC though it uses external AR8033 via SGMII but it actually works.
In your topo, you can just trust the qca8337 as a managed switch. But the current framework doesn't support such topo. There is no way to describe a attacted phy connect to a DSA switch by MDI.
IPQ5018 MAC - IPQ5018 PHY ---MDI--- QCA8337 PHY - QCA8337 MAC
For now, what i suggest is just ignore qca8337. Use the uboot initialization, which will configure it as a dumb switch.
thanks @hzyitc. If we do want to make it work as a DSA switch, can you give some pointers as to what needs to be changed? Is it just the qca8k driver, or also the underlying linux DSA code, and perhaps even QCA nss-dp?
@robimarko: at mentioned you as I saw it mentioned in another thread where you were working on a solution for the following:
[90297.444117] nss-dp 39d00000.dp2 wan: cannot get VSI ID for port 2
[90306.404102] nss-dp 39d00000.dp2 wan: cannot get VSI ID for port 2
[90326.884156] nss-dp 39d00000.dp2 wan: cannot get VSI ID for port 2
[90358.884109] nss-dp 39d00000.dp2 wan: cannot get VSI ID for port 2
[90422.244124] nss-dp 39d00000.dp2 wan: cannot get VSI ID for port 2
[90451.684096] nss-dp 39d00000.dp2 wan: cannot get VSI ID for port 2
Did you ever get to a solution for this roaming issue?
