Adding support for MikroTik RouterBOARD 711G 5HnD

For Developers
#1

I'm trying to port OpenWrt to a RouterBOARD 711G 5HnD.
I have never ported any device before. I found some instructions in the wiki that helped me compile OpenWrt and boot the initramfs image over the network using MikroTik's Etherboot mode. I then managed to make the NAND flash chip, wifi and ethernet work. I'm using the deprecated ar71xx target since that's what the instructions in the wiki suggested and I couldn't even get an image for the ath79 target booting after several hours of trying.
Everything works as expected if I Etherboot the image but if I use sysupgrade to flash the image and boot off of the flash chip, ethernet stops working correctly.
Here is the output of ip a if I boot off of the network:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br-lan state UP qlen 1000
    link/ether d4:ca:6d:42:ad:4b brd ff:ff:ff:ff:ff:ff
4: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
    link/ether d4:ca:6d:42:ad:4b brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.149/24 brd 192.168.2.255 scope global br-lan
       valid_lft forever preferred_lft forever
    inet6 fe80::d6ca:6dff:fe42:ad4b/64 scope link 
       valid_lft forever preferred_lft forever
5: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-lan state UP qlen 1000
    link/ether d4:ca:6d:42:ad:4a brd ff:ff:ff:ff:ff:ff
    inet6 fe80::d6ca:6dff:fe42:ad4a/64 scope link 
       valid_lft forever preferred_lft forever

Here is the output of ip a if I boot off the NAND flash chip:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br-lan state UP qlen 1000
    link/ether d4:ca:6d:42:ad:4b brd ff:ff:ff:ff:ff:ff
4: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
    link/ether d4:ca:6d:42:ad:4b brd ff:ff:ff:ff:ff:ff
    inet6 fe80::d6ca:6dff:fe42:ad4b/64 scope link 
       valid_lft forever preferred_lft forever
5: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-lan state UP qlen 1000
    link/ether d4:ca:6d:42:ad:4a brd ff:ff:ff:ff:ff:ff
    inet6 fe80::d6ca:6dff:fe42:ad4a/64 scope link 
       valid_lft forever preferred_lft forever

As you can see, br-lan has no IPv4 address. However, the ethernet port still seems to work somewhat as the DHCP server logs show DHCPDISCOVER messages every couple seconds (slightly random intervals). The ethernet indicator LED on the RouterBOARD and the LED on the switch it's attached to blink when that happens. If I configure OpenWrt to act as a DHCP server, as is the default, then attached devices won't get an IP address assigned.
I have no idea why this happens as I'm booting basically the same image both times, only once via ethernet and once via the onboard storage.

Has anyone seen this behaviour before or knows what could be going on? I have tried everything I could think of for multiple days now and am getting desperate.

I'm happy to provide any additional information that may be useful for debugging this.

#2

I still haven't managed to boot from flash but my best theory as to why this is happening is that the AR8021 Gigabit Ethernet PHY chip is initialised by the boot loader when booting from the network but not when booting from flash and the kernel is not initialising it properly. With the help of AR8021 phy not working in gigabit mode; how to configure pll-data cells? I tried to extract the pll-data after booting from the network and then applying it on boot from flash but ultimately I couldn't get it to work. Maybe I just did something wrong so the pll-data wasn't set correctly or some other configuration for the chip is missing.

#3

Which SoC is in this device? I could not find any information on this board, but e.g. on AR72xx I couldn't make Gigabit Ethernet work either.

But if it works when initialised by the bootloader, it's probably not an issue with GMAC PLL settings, since these registers are overwitten whenever a new link speed is detected.
Does 100BASE-T work when it is enforced by the link partner?

Are you still using ar71xx? Consider that this target has been removed by now, what were the issues you encountered when building for ath79?

#4

It's an Atheros AR7242.

I think I tested this and it still didn't work when I forced 100BASE-T using a managed switch. But I'm not 100% sure anymore, it's been a couple months. The router is currently in operation on my roof, so testing it again would be inconvenient.

Yes. I'm also still using some 19.07 version of OpenWrt that was the latest at the time.

Yes, I know and it would be great to run the latest version (and is mandatory for getting support for this router into OpenWrt) but I couldn't get it to work. Building for the ar71xx target was relatively easy. I found this wonderful wiki page, which contains a patch. I then adapted the patch slightly to make it apply on the latest 19.07 OpenWrt release and compiled it. Booting the resulting image over the network worked on the first try I think and I could see some boot messages on the serial port. Then I could slowly make features like networking and access to the NAND flash chip work with the help of the serial port. But when I tried building and booting an image for ath79 using the latest OpenWrt master at the time, the bootloader wouldn't accept the image and simply reboot. The same behaviour happens when the image is too large, so I removed packages until the image was much smaller than an image that I knew would boot, without success. I tried various things I can't remember now and the pll thing but nothing worked. I never got any output on the serial port. Unfortunately the bootloader is completely silent on the serial port, so I couldn't see what was going on, which made it impossible for me to debug this.

#5

I have a MikroTik RouterBOARD 711 5HnD (so without the G) on the other side of the wireless link which is much easier to access (not on a roof). The main difference between the two is that the one without the G only has 100MBit Ethernet using a single port of the switch built into the AR7241 SOC. Booting from flash totally works on that device. But for trying to port the it to ath79, this router should be equally good. So if you have any ideas on what else I could try to make ath79 work, I'd be very interested in hearing them.

#6

This sounds a lot easier to support for a beginning, when the internal switch can be used. Actually, from my experience, even ath9k wifi is much simpler to get working than ethernet, so for new devices with complicated ethernet configurations, I usually have wifi enabled by default.
This can be done using a uci script with fixed values for ESSID and PSK. Here is an example I used for DAP-1320 (50_wifi_defaults.sh, even reading both values from the flash) :

Most important thing would be to find the flash layout, but in this case it seems to be available and verified (regarding the position of art) already from ar71xx.

Since AR7241 has no built-in wifi, the pcie node in the dts needs be enabled (status="okay"); I'm not quite sure whether the wifi node is also needed to include the ath9k driver in the image, but it should look something like this, the VID of the device (002e) might be different for the 5GHz radio though:

&pcie {
	status = "okay";

	ath9k: wifi@0,0 {
		compatible = "pci168c,002e";
		reg = <0x0000 0 0 0 0>;
		#gpio-cells = <2>;
		gpio-controller;
		qca,no-eeprom;
	};
};

If there is a valid MAC address at 0x1002 in art, it should be found and used automatically... otherwise, this can be fixed later, first of all access to the device should be possible :wink:
Most of the ethernet configuration should be handled automatically in ath79, but let me know when you encounter any troubles.

#7

Ok, so I took town the wireless link and tried to make the RouterBOARD 711 5HnD boot the latest OpenWrt master again. I created the following patch:

diff --git a/target/linux/ath79/dts/ar7241_mikrotik_routerboard-711-5hnd.dts b/target/linux/ath79/dts/ar7241_mikrotik_routerboard-711-5hnd.dts
new file mode 100644
index 0000000000..7a2a7de671
--- /dev/null
+++ b/target/linux/ath79/dts/ar7241_mikrotik_routerboard-711-5hnd.dts
@@ -0,0 +1,12 @@
+// SPDX-License-Identifier: GPL-2.0-or-later OR MIT
+
+#include "ar7241.dtsi"
+
+/ {
+	compatible = "mikrotik,routerboard-711-5hnd", "qca,ar7241";
+	model = "MikroTik RouterBOARD 711 5HnD";
+
+	chosen {
+		bootargs = "earlycon=ns16550a,mmio,0x18020000 console=ttyS0,115200 earlyprintk";
+	};
+};
diff --git a/target/linux/ath79/image/mikrotik.mk b/target/linux/ath79/image/mikrotik.mk
index 2324ecc195..94f40d65b1 100644
--- a/target/linux/ath79/image/mikrotik.mk
+++ b/target/linux/ath79/image/mikrotik.mk
@@ -1,5 +1,21 @@
 include ./common-mikrotik.mk
 
+define Device/mikrotik_routerboard-711-5hnd
+  $(Device/mikrotik_nand)
+  SOC := ar7241
+  DEVICE_MODEL := RouterBOARD 711 5HnD
+  SUPPORTED_DEVICES += rb-711-5hnd
+endef
+TARGET_DEVICES += mikrotik_routerboard-711-5hnd
+
 define Device/mikrotik_routerboard-493g
   $(Device/mikrotik_nand)
   SOC := ar7161
diff --git a/target/linux/ath79/mikrotik/base-files/etc/board.d/02_network b/target/linux/ath79/mikrotik/base-files/etc/board.d/02_network
index 4058742133..a2e0ccf61c 100644
--- a/target/linux/ath79/mikrotik/base-files/etc/board.d/02_network
+++ b/target/linux/ath79/mikrotik/base-files/etc/board.d/02_network
@@ -14,6 +14,8 @@ ath79_setup_interfaces()
 		ucidef_add_switch "switch1" \
 			"0@eth1" "1:lan:4" "2:lan:1" "3:lan:2" "4:lan:3"
 		;;
+	mikrotik,routerboard-711-5hnd|\
 	mikrotik,routerboard-912uag-2hpnd|\
 	mikrotik,routerboard-lhg-2nd|\
 	mikrotik,routerboard-sxt-5nd-r2|\
@@ -36,6 +38,8 @@ ath79_setup_macs()
 	local mac_base="$(cat /sys/firmware/mikrotik/hard_config/mac_base)"
 
 	case "$board" in
+	mikrotik,routerboard-711-5hnd|\
 	mikrotik,routerboard-912uag-2hpnd|\
 	mikrotik,routerboard-lhg-2nd|\
 	mikrotik,routerboard-sxt-5nd-r2|\
diff --git a/target/linux/ath79/mikrotik/base-files/lib/upgrade/platform.sh b/target/linux/ath79/mikrotik/base-files/lib/upgrade/platform.sh
index 6962c6fdcc..d08ef6b7ef 100644
--- a/target/linux/ath79/mikrotik/base-files/lib/upgrade/platform.sh
+++ b/target/linux/ath79/mikrotik/base-files/lib/upgrade/platform.sh
@@ -31,6 +31,8 @@ platform_do_upgrade() {
 	local board=$(board_name)
 
 	case "$board" in
+	mikrotik,routerboard-711-5hnd|\
 	mikrotik,routerboard-493g|\
 	mikrotik,routerboard-912uag-2hpnd|\
 	mikrotik,routerboard-921gs-5hpacd-15s|\

I executed make menuconfig and removed all the packages to make the resulting image as small as possible. As the first step, I only want to boot the kernel and don't care about userspace at all. The device tree is deliberately minimal, not containing anything related to wifi, ethernet, flash, etc. as this doesn't help me get over this first hurdle.
Unfortunately the bootloader doesn't seem to accept the image I built, as the router reboots just after downloading the image via the network. This is the same behaviour as when trying to boot an image that is too large. Since the bootloader is completely silent on the serial port, this makes it extremely difficult to debug what the heck is going on. When I first tried this a couple months ago, I even tried to flash the image to NAND from a running instance of OpenWrt 19.x, which also didn't work, so I don't think booting over ethernet is the problem here.
While the tips you gave me may prove very helpful in the future, they unfortunately don't help me at all with this first hurdle.

#8

Should I try to replace the vendor bootloader with u-boot?
I'm just a little scared of bricking the board. I read that there is a way to recover it and the board has an SPI flash chip that presumably holds the backup bootloader, but still.
I've never compiled u-boot before. Any tipps for what I need to watch out for and what I need to adjust in u-boot to hopefully be compatible with the board?