Add support for MikroTik RB5009UG

Borromini · September 18, 2022, 2:35pm

@Arie Did you look at robimarko's tree? He has 5.15 but had quite a few rough edges, admittedly (and I have no idea which ones). But might help.

Arie · September 18, 2022, 2:43pm

Ah nice, I hadn't seen that one

Borromini · September 18, 2022, 2:47pm

All I have added to my 5.15 tree is this patch for the Amethyst switch, this for the kernel config:

diff --git a/target/linux/mvebu/cortexa72/config-5.15 b/target/linux/mvebu/cortexa72/config-5.15
index 7e07c28ba6..b4541413db 100644
--- a/target/linux/mvebu/cortexa72/config-5.15
+++ b/target/linux/mvebu/cortexa72/config-5.15
@@ -34,6 +34,7 @@ CONFIG_ARM_GIC_V3_ITS=y
 CONFIG_ARM_GIC_V3_ITS_PCI=y
 # CONFIG_ARM_PL172_MPMC is not set
 CONFIG_ARM_PSCI_FW=y
+CONFIG_ARM_SBSA_WATCHDOG=y
 CONFIG_AUDIT_ARCH_COMPAT_GENERIC=y
 CONFIG_CC_HAVE_STACKPROTECTOR_SYSREG=y
 CONFIG_DMA_DIRECT_REMAP=y
@@ -52,6 +53,9 @@ CONFIG_MFD_IEI_WT61P803_PUZZLE=y
 CONFIG_MFD_SYSCON=y
 CONFIG_MMC_SDHCI_XENON=y
 CONFIG_MODULES_USE_ELF_RELA=y
+CONFIG_MIKROTIK=y
+CONFIG_MIKROTIK_RB_SYSFS=y
+CONFIG_MTD_SPLIT_FIT_FW=y
 CONFIG_MVEBU_GICP=y
 CONFIG_MVEBU_ICU=y
 CONFIG_MVEBU_ODMI=y

The MikroTik bits and split FIT firmware symbol are needed for how adrons set up his image (I suppose you are using his hacked RouterBOOT).

Then this patch to backport QCA8081 PHY support. Then this to enable the QCA8081 NIC in 5.15:

diff --git a/target/linux/mvebu/cortexa72/config-5.15 b/target/linux/mvebu/cortexa72/config-5.15
index b4541413db..35910cac7e 100644
--- a/target/linux/mvebu/cortexa72/config-5.15
+++ b/target/linux/mvebu/cortexa72/config-5.15
@@ -35,6 +35,7 @@ CONFIG_ARM_GIC_V3_ITS_PCI=y
 # CONFIG_ARM_PL172_MPMC is not set
 CONFIG_ARM_PSCI_FW=y
 CONFIG_ARM_SBSA_WATCHDOG=y
+CONFIG_AT803X_PHY=y
 CONFIG_AUDIT_ARCH_COMPAT_GENERIC=y
 CONFIG_CC_HAVE_STACKPROTECTOR_SYSREG=y
 CONFIG_DMA_DIRECT_REMAP=y

I keep both config-5.15 modifications as separate commits. Robimarko's tree uses the 'qca8081' label so he has extra edits to the DTS, but adrons just sticks to the generic 'switch0phy9'.

I have uploaded a patch set here to play with. Mind you, this merely backports onto the present master tree what seems to be necessary. I haven't confirmed this actually works (but there's little chance it makes things worse ).

Borromini · September 20, 2022, 8:49pm

Well I am a happy camper. @robimarko You'll be happy to hear that with the QCA8081 backports and 5.15, 2,5 Gbps seems to be working

I haven't tested stability or anything, this was just booting a freshly compiled image with the patch linked to above. Dmesg is squeaky clean, no errors.

Dmesg:

root@OpenWrt:~# logread -e "f212a200.mdio-mii:00 p1"
Fri Aug  1 12:19:07 2098 kern.info kernel: [    3.734769] mv88e6085 f212a200.mdio-mii:00 p1 (uninitialized): PHY [!cp0!config-space@f2000000!mdio@12a200!switch@0!mdio1:00] driver [Qualcomm QCA8081] (irq=POLL)
Fri Aug  1 12:19:07 2098 kern.info kernel: [   12.416333] mv88e6085 f212a200.mdio-mii:00 p1: configuring for inband/sgmii link mode
Fri Aug  1 12:19:09 2098 kern.info kernel: [   19.234765] mv88e6085 f212a200.mdio-mii:00 p1: configuring for inband/sgmii link mode
Fri Aug  1 12:19:13 2098 kern.info kernel: [   23.091929] mv88e6085 f212a200.mdio-mii:00 p1: Link is Up - 2.5Gbps/Full - flow control rx/tx

Kernel:

root@OpenWrt:~# uname -a
Linux OpenWrt 5.15.68 #0 SMP Sun Sep 18 15:29:21 2022 aarch64 GNU/Linux

Ethtool with a Realtek RTL8125 (rev 4) at the other end:

Summary

root@OpenWrt:~# ethtool p1
Settings for p1:
	Supported ports: [ TP	 MII ]
	Supported link modes:   10baseT/Half 10baseT/Full
	                        100baseT/Half 100baseT/Full
	                        1000baseT/Full
	                        1000baseX/Full
	                        2500baseT/Full
	Supported pause frame use: Symmetric
	Supports auto-negotiation: Yes
	Supported FEC modes: Not reported
	Advertised link modes:  10baseT/Half 10baseT/Full
	                        100baseT/Half 100baseT/Full
	                        1000baseT/Full
	                        1000baseX/Full
	                        2500baseT/Full
	Advertised pause frame use: Symmetric
	Advertised auto-negotiation: Yes
	Advertised FEC modes: Not reported
	Link partner advertised link modes:  10baseT/Half 10baseT/Full
	                                     100baseT/Half 100baseT/Full
	                                     1000baseT/Full
	                                     2500baseT/Full
	Link partner advertised pause frame use: Symmetric Receive-only
	Link partner advertised auto-negotiation: Yes
	Link partner advertised FEC modes: Not reported
	Speed: 2500Mb/s
	Duplex: Full
	Auto-negotiation: on
	master-slave cfg: preferred slave
	master-slave status: slave
	Port: Twisted Pair
	PHYAD: 0
	Transceiver: external
	MDI-X: off (auto)
	Supports Wake-on: g
	Wake-on: d
	Link detected: yes

This basically means that, beyond the bootloader conundrum, the RB5009UG will be fully supported with kernel 5.15!

robimarko · September 20, 2022, 9:15pm

Is the traffic going through on 2.5G?

Cause, I think it would link up but not traffic would pass

Borromini · September 20, 2022, 9:17pm

I am typing this message connected through the QCA8081 as we speak .

robimarko · September 20, 2022, 9:17pm

Great, then it got magically fixed

Borromini · September 20, 2022, 9:20pm

I'm on adron's DTS, but differences with yours seem to be minimal, probably to the point of being irrelevant in this case.

Arie · September 20, 2022, 9:28pm

This is just with your patch set?

Borromini · September 20, 2022, 9:29pm

Yep, it is.

@adron Is the strict-cpu-mode patch still necessary for 5.15? I noticed it's not in robimarko's tree and I initially overlooked it when checking what needed to be patched in for 5.15.

$ cat target/linux/mvebu/patches-5.10/708-net-dsa-strict-cpu-mode.patch 
--- a/include/net/dsa.h
+++ b/include/net/dsa.h
@@ -214,6 +214,7 @@ struct dsa_port {
 	struct device_node	*dn;
 	unsigned int		ageing_time;
 	bool			vlan_filtering;
+	bool			strict_cpu_mode;
 	u8			stp_state;
 	struct net_device	*bridge_dev;
 	struct devlink_port	devlink_port;
--- a/net/dsa/tag_dsa.c
+++ b/net/dsa/tag_dsa.c
@@ -131,8 +131,10 @@ static struct sk_buff *dsa_rcv(struct sk
 			skb->data - ETH_HLEN - DSA_HLEN,
 			2 * ETH_ALEN);
 	}
-
-	skb->offload_fwd_mark = 1;
+	if (likely(dev->dsa_ptr->strict_cpu_mode))
+		skb->offload_fwd_mark = 0;
+	else
+		skb->offload_fwd_mark = 1;
 
 	return skb;
 }

adron · September 21, 2022, 2:41am

In fact, only I need this mode, you don't need it. It disables Mac Learning on all ports and forces all traffic to go through the CPU. This mode is useful if you need to plug two DSA ports into the same switch.

snh · September 23, 2022, 10:53am

Firstly, amazing work everyone, it is so exciting to see the progress here over the past year, especially the recently developments around 5.15.

@Borromini Is @adron's commit with the DTS the only other commit that needs to be added to your patch set, or is there more required?

I know there is still the bootloader challenge, but is there a chance we'll see these changes merged into the OpenWrt project regardless? Or does the requirement for a patched bootloader preclude support being officially added, even with the caveat of the bootloader being an external dependency?

Borromini · September 23, 2022, 11:01am

You can use my earlier patch set here then apply the 5.15 patch set on top. And take a close look at the config adrons used to compile.

Might be it does not apply cleanly to master anymore, in that case let me know.

AFAICT merge policy is not to depend on external binaries. Which is still the case here, until someone can provide a compatible u-boot for this beauty.

Borromini · September 23, 2022, 1:35pm

@snh I was sure I had it floating around somewhere, and I had indeed added it to the wiki.

You can grab the most recent patch set for master there, then apply the 5.15 patch set on top. I will add that to the wiki as well later.

Arie · September 23, 2022, 9:12pm

I keep ending up with a router that won't come back online with that set of patches, on 5.15, with my old config.
Would you mind sharing a .config that works for you, maybe I'm missing something.

I have:

Checked out master openwrt
Used git am 381333 on that https://0paste.com/381333#hl patch file
Used git am 0001-mvebu-rb5009ug-support.patch on your patch file ( https://volatilesystems.org/dl/lede/targets/patches/RB5009UG%2BS%2BIN/0001-mvebu-rb5009ug-support.patch )
Built the kernel based on a previous working .config, just enabling the testing kernel: https://gist.github.com/Arie/96df66725530d950afdb38a406db5411

Happy on 5.10 for now, but would really like to try 5.15

Borromini · September 23, 2022, 9:15pm

Looks like I might have missed the Amethyst patch in my set, I'm rebuilding it. Sorry for the hassle.

Borromini · September 23, 2022, 10:33pm

OK, I've revised the patch set and also changed the default scheduler to schedutil like in Robimarko's tree (I believe this is a 5.15 thing that did not apply to 5.10). I just built and run-tested this on my own RB5009UG+S+IN, tested QCA8081 again. Working fine.

@Arie I've upped my config as well, you can take a peek, lots of personal additions (unbound etc.) so be sure to strip it to our taste.

I've split up the patches between 22.03 and master, you can find them here again. I've linked to them in the wiki as well.

Arie · September 25, 2022, 3:52pm

Thanks a lot, I managed to create an image with those patches on master that booted, at least for a little while. After a minute or so the router seems to crash and reboot. (I use a RB5009UPr+S+IN btw)
Despite logread -f and spamming dmesg I have no hints as to what happens. Both on sysupgrade with keeping the config and starting with a clean one, this behavior is the same.

Is that something you experienced before? Else I guess I'll set up UART sometime.

Borromini · September 25, 2022, 4:06pm

No idea what MikroTik revised for the PoE version. Can only speak for the original non PoE version. Haven't kept it up long either though.

Arie · September 25, 2022, 4:11pm

Yeah me neither, the 5.10 versions just work on the PoE version.

The only hint of trouble was that applying the final patch set didn't go cleanly:

git am 0000-2-rb5009ug-master-5.15.patch

Applying: mvebu: cortexa72: 5.15: schedutil as default scheduler
error: patch failed: target/linux/mvebu/cortexa72/config-5.15:37
error: target/linux/mvebu/cortexa72/config-5.15: patch does not apply
Patch failed at 0001 mvebu: cortexa72: 5.15: schedutil as default scheduler

After solving that one manually and running git am --continue:

Applying: mvebu: rb5009: add QCA8081 PHY support for 5.15
error: patch failed: target/linux/mvebu/cortexa72/config-5.15:35
error: target/linux/mvebu/cortexa72/config-5.15: patch does not apply
Patch failed at 0004 mvebu: rb5009: add QCA8081 PHY support for 5.15

Looking at the diffs after the two failing patches, I think my target/linux/mvebu/cortexa72/config-5.15 is different from what your patch was for, e.g. there's no CONFIG_ARM_SBSA_WATCHDOG=y in mine