I see that there are no changes for ath79 target here - do you think that copying image recipe from either ipq40xx or ramips should work? I'll be happy to test that.
Thank you. My dirty python decoder worked as expected on that file, it was different to the first, and matched the emulated uncompressed file, so the block format I found stands for now.
For NOR targets, yes, it is worth testing, will need to make it match the mikrotik nor recipe in common-mikrotik. I have not looked at ath79 NAND (not planning to).
I initially avoided it for the big endian. I remember seeing some data endian converted by RouterBOOT on MIPSBE, but I don't remember what… Hopefully they do the same for the NPK.
I can test this later as well, but feel free to try.
Ok, I'll try that soon (hopefully).
As for NAND, it seems, that RouterBOOT v7 can actually boot current image format as is. So there is nothing much left to do with that regard, maybe save for inventing a better method, than generating dummy yaffs section for kernel, which was the reason for disabling NAND devices in the first place.
This is great news.
There are two questions in my mind:
Can you please create a pull request for the "routerboot-v7" branch of yours, so it can get to Openwrt and we can start adding device support based on that? I am testing it on two devices already and it works, so we can add "tested by" tags right away.
On the LZ77 part: I assume the platform driver cannot really use a python script, so I wonder whats next? There are two devices which are ready for Openwrt, only this part is missing for them to be fully functional.
I got a chance to test this, and my ath79 wapg with RouterBOOT 6.45.8 boot the bootimage NPK file-container kernel fine. I don't know how far back RouterBOOT started to be able to load bootimage NPK file-container kernel, but the backup booter 3.41 worked here as well.
RouterBOOT backup booter 3.41 RouterBOARD wAP G-5HacT2HnD CPU frequency: 720 MHz Memory speed: 300 MHz Memory size: 64 MiB Storage size: 16 MiB Press any key within 2 seconds to enter setup.. loading kernel... OK setting up elf image... OK jumping to kernel code [ 0.000000] Linux version 5.15.98 (john@john) (mips-openwrt-linux-musl-gcc (OpenWrt GCC 12.2.0 r21569+48-9b6dd57a33) 12.2.0, GNU ld (GNU Binutils) 2.40.0) #0 Sat Mar 18 06:54:53 2023 [ 0.000000] printk: bootconsole [early0] enabled [ 0.000000] CPU0 revision is: 00019750 (MIPS 74Kc) [ 0.000000] MIPS: machine is MikroTik RouterBOARD wAP G-5HacT2HnD [ 0.000000] SoC: Qualcomm Atheros QCA9556 ver 1 rev 0 [ 0.000000] Initrd not found or empty - disabling initrd
root@OpenWrt:~# hexdump -C -n 64 /dev/mtd$(find_mtd_index kernel) 00000000 00 00 00 01 00 00 00 01 ff ff 62 6f 6f 74 69 6d |..........bootim| 00000010 61 67 65 00 00 00 00 00 00 00 00 00 00 00 00 00 |age.............| 00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................| * 00000040
urgh, no way around this if we want to directly boot kernel from RouterBOOT. ipq40xx NAND (UBI) are the only Mikrotik devices I have seen which do not use YAFFS.
Both of these are proof-of-concept rather than a clean tool.
Maybe you could convince a maintainer that you need this whole python framework to build the NPK, but ideally someone creates a new tool to do it. If you wanted to use this as-is, you would need to get
python3-venv added to the buildbots for it, but first tests should be added for the additional functionality I tacked onto the framework, and that sent upstream for code review.
Someone extends rb_hardconfig
hc_wlan_data_unpack with the logic to decompress the Mikrotik LZ77 block format.
Anyone is welcome to try either or both of these things…
To be honest, I thought that at least the general routerboot-v7 parts can be just pulled in. I did that on my branch with the Chateau 12 and Hap ac3 and it just worked without any issues or hacking. I did not realized that is also proof of concept, like the LZ77 part.
On the LZ77 part, I do hope that you or someone more capable than me might pick this up and make the platform driver extension. There are some devices already which only needs this to be fully functional on Openwrt, like the Chateau 12LTE or the Chateau 5G.
whats the root password, it boots up on 192.168.10.1 but it's asking for a root password
For the record, I've just installed OpenWRT 22.03.5 on a new cAP AC, without much in the way of a hitch...
I did not even have to persuade the RB to boot via bootp for me, it just did that "out of the box".
I just modded my entry in ISC dhcpd.conf to let it know the MAC address of my new cAP AC, so that when a bootrequest comes, the correct "next server" and "filename" gets served, and the RB downloads the OpenWRT initramfs image via TFTP (hpa-tftpd here). From there, I flashed the squashfs image without a problem.
Apparently the cAP AC is based on the IPQ4018, and the flash chip gets detected by the spi-nor driver - i.e. it's a serial NOR flash, as opposed to a NAND device... if that makes any difference.
In the device today, I've noticed the following version of RouterBoot:
cat /sys/firmware/mikrotik/hard_config/booter_version 6.48.6
Based on the big red warning in the common TOH entry for Mikrotik devices, about RouterOS 7, the RouterOS 6 series starting from 6.46 is supposedly a no-no just as well. Anything past 6.45.8 actually.
I'm looking at another Mikrotik machine here, an hAP AC2. This one's got RouterBoot 6.45.9. And another piece of the cAP AC, with 6.47.9. Both of them historically flashed OpenWRT without a problem.
Exactly where is the dividing line? Should I expect trouble, once I happen to get my hands on a cAP AC with RouterOS v7 ex works in the future? Or is this whole IPQ4018-based family exempt from that fat red warning? In my context, the red warning feels misleading / not detailed enough...
Looks like that wiki page has been added to far to many times without anyone doing cleanup. Urgh, what a mess.
From what I have seen:
RouterBOOT v7 no longer works with the historic YAFFS ELF kernel boot. It wants YAFFS NPK bootimage boot, which OpenWRT does not do (yet).
ipq40xx NAND works, because it uses UBI to boot.
Aside from that, on some very old hardware, there is not enough RAM for RouterBOOT to be able to load an initramfs image, and a few devices can not boot an installed kernel either.
There are many versions of RouterBOOT where DHCP boot is broken: don't use it.
Someone said they had a hap ac3 which had LZ77 wlan data, but could seemingly not be bothered to share the data to get it to work: No Wireless Mikrotek RBD53iG-5HacD2HnD - #3 by digga106
I have a Hap Ac3 LTE6 kit (very same base board), but both the Hap Ac3 and the Hap Ac3 LTE6 kit are ROS6 devices, so it would be strange to have a 77ZL version from it, as netinstall allows them to be downgraded to ROS v6 images, which would fail if the hard_config is LZ77 based...
With that said, if you have some time to upgrade the platform driver with the LZ77 part (not sure how the ROS v7 boot part is atm), I could instantly add two devices to Openwrt ipq6xxx will also have support at some point in time, those devices will also benefit if we can boot Openwrt on them as well
The LZ77 packed wlan hardcfg tag has been around long before RouterOS / RouterBOOT v7:
[john@john _routeros-arm-6.46.npk.extracted]$ grep -H -r --binary-files=text -r -i lz77 $(find . -iname 'flash.ko') ./squashfs-root/lib/modules/3.3.5/misc/flash.ko:flash ERROR: radio data lz77 decompress failed: %d ./squashfs-root/lib/modules/3.3.5/misc/flash.ko:radio data lz77 decompressed from %d to %d
Not yet, still somewhere in my todo list…
Just to keep awareness on this one.
@johnth in case you want me to test anything, please let me know.
very rough, expect test device to burn…
probably simple mistakes aplenty.
Would have been sooner with at test device (will code for hardware), ha.
Not a problem, I have a complete flash dumb I can rewrite to the device
I can do tests on monday, will get back to you.
I applied your commit but nothing happens.
Can it be that the
target/linux/ipq40xx/base-files/etc/hotplug.d/firmware/11-ath10k-caldata part needs some modifications for this to work?
root@cpe00005:~# ls /lib/firmware/ath10k/QCA4019/hw1.0/ firmware-5.bin
No cal nor board files are present after boot where they should be.
Relevant part of dmesg:
[ 0.958302] MikroTik RouterBOARD hardware configuration sysfs driver v0.08 [ 0.961324] MikroTik RouterBOARD software configuration sysfs driver v0.05 ..... [ 18.456292] ath10k_ahb a000000.wifi: qca4019 hw1.0 target 0x01000000 chip_id 0x003b00ff sub 0000:0000 [ 18.456366] ath10k_ahb a000000.wifi: kconfig debug 0 debugfs 1 tracing 0 dfs 1 testmode 0 [ 18.468302] ath10k_ahb a000000.wifi: firmware ver 10.4b-ct-4019-fW-13-5ae337bb1 api 5 features mfp,peer-flow-ctrl,txstatus-noack,wmi-10.x-CT,ratemask-CT,regdump-CT,txrate-CT,flush-all-CT,pingpong-CT,ch-regs-CT,nop-CT,set-special-CT,tx-rc-CT,cust-stats-CT,txrate2-CT,beacon-cb-CT,wmi-block-ack-CT,wmi-bcn-rc-CT crc32 6b2b5c5b [ 18.906493] ath10k_ahb a000000.wifi: failed to fetch board-2.bin or board.bin from ath10k/QCA4019/hw1.0 [ 18.906578] ath10k_ahb a000000.wifi: failed to fetch board file: -12 [ 18.915032] ath10k_ahb a000000.wifi: could not probe fw (-12) [ 19.918950] ath10k_ahb a800000.wifi: qca4019 hw1.0 target 0x01000000 chip_id 0x003b00ff sub 0000:0000 [ 19.919046] ath10k_ahb a800000.wifi: kconfig debug 0 debugfs 1 tracing 0 dfs 1 testmode 0 [ 19.932464] ath10k_ahb a800000.wifi: firmware ver 10.4b-ct-4019-fW-13-5ae337bb1 api 5 features mfp,peer-flow-ctrl,txstatus-noack,wmi-10.x-CT,ratemask-CT,regdump-CT,txrate-CT,flush-all-CT,pingpong-CT,ch-regs-CT,nop-CT,set-special-CT,tx-rc-CT,cust-stats-CT,txrate2-CT,beacon-cb-CT,wmi-block-ack-CT,wmi-bcn-rc-CT crc32 6b2b5c5b [ 20.360707] ath10k_ahb a800000.wifi: failed to fetch board-2.bin or board.bin from ath10k/QCA4019/hw1.0 [ 20.360799] ath10k_ahb a800000.wifi: failed to fetch board file: -12 [ 20.369294] ath10k_ahb a800000.wifi: could not probe fw (-12)
Will need to either build with
CONFIG_DYNAMIC_DEBUG, then enable dynamic debug for
rmmod ath10k_pci; modprobe ath10k_pci, or change the newly added
pr_debug calls under
pr_info so we can get some more details.
I can push a commit that is more verbose by default for now.
Edit: Think this is because I am passing the DRE\x00 magic to tag_find. Will push that
It works! Both radios are up and running, the WLAN MACs are also correct, the wireless driver is loading without any complaints. FYI, this is also a device on Routerboot_v7, image is with your patch applied:
ipq40xx: MikroTik RouterBOOT v7 NOR specific changes to generate compatbile images, adapted to ARM traget.
Do you still need the debug logs you described? Or anything else that might help? Let me know.
Great, thank you. We only got to this stage because you provided the mtd dump for hard tags.
Not any more. They would be nice to have, to double check that things are working as expected, but not needed.
If we also debug the lz77 decompress file, I think we would run out of dmesg space, so would need to change kernel config
CONFIG_LOG_BUF_SHIFT, or kernel param
log_buf_len=n (bytes, must equal a power of 2).
Can we do anything else? I tested it on multiple Chateau 12 devices they all work. If you need anything further, please let us know.