Support for Mikrotik RB3011UiAS-RM?

Well, like for Wireless Wire dish BDFs are missing.
Thats because we have no idea with what kind of protection are they protecting them in /bndl/wireless/firmware/

This is what is severely limiting the Wireless Wire Dish performance.

Files are not compressed, but rather protected with some coding.
This also hits ath10k as pretty much all IPQ40XY devices use non standard BDFs for calibration.

I have been trying to dump the whole RAM and possibly find them there as strace revelead that wireless binary only loads them in RAM and them insmods the driver.
But LiMe fails to dump whole RAM as 3.3.5 driver has bug in which kernel will return completely unrealisticly high memory end address.

The .brd file crypto appears to be part of the wireless binary.

It's C++ though, and I don't understand a few bits. And I'm not sure whether that's all.

  • There's a hardcoded abcdABCDqwerty string which seems to be used in in a key derivation scheme (1024 rounds of SHA256?). Seems like there might be a prefix, but I can't tell what it is.

  • If you find the sole xref to that string, right after it performs a read file followed by a LFSR-based xor crypto loop. Key doesn't seem to be involved, so I must be missing something here.

If you have a working gdb and are able to make some core dumps of wireless at the right offsets that might help. The only thing I don't know at all is how difficult it is to get it to load the .brd file while having gdb attached (kill it and hope you can respawn it with gdb attached, or just start it again).

If you have gdb, a firmware version and hash of wireless I can look up some breakpoint offsets..

Yes, I have already run GDB couple of times and can make dumps or a core dump.
Well, according to strace wireless binary will load both the firmware and .brd and .msg files every time that you re-enable wireless interface as when I decompile you can clearly see it will rmmod the driver every time it's disabled and insmod after loading firmware and board files.

I can provide you with whatever you need as I have root shell.

SHA256 of wireless 6.45.6: a125ed0b31fb28b863ae2806f25fd37dba41474ae0f2baca0f73dce9387fba91

  • Attach gdb
  • Set breakpoints using b *0x<address> for all addresses shown below: e.g. b *0x000997ec
  • Continue: c
  • For every breakpoint hit: generate-core-file <address>.core
  • The 0x000997ec bp will hit 1024 times, just 'c 1023' after the first hit or delete it

Adresses:

SHA256 update:

000997ec 84 de fd eb     bl update

Read .brd file:

0009988c 52 dc fd eb     bl readFile

Call to crypto fn + return address:

000998b0 df 57 fe eb     bl FUN_0002f834
000998b4 b8 30 d4 e5     ldrb r3,[r4,#0xb8]

Well it looks like wireless binary actually does not decrypt the files.
I have added the breakpoints but none have been hit.
Maybe I am doing something wrong as I cant get any breakpoint hits, I only managed to get one hit one insmodding the kmod but I somehow managed to stop the process and then started it again with gdb.
Done it twice and insmod gets hit but no hit on crypto functions.
I ran strace on wireless binary and it looks like it only loads the file.

open("/dev/flash", O_RDWR) = 24
ioctl(24, _IOC(_IOC_READ, 0x46, 0x36, 0x20), 0x7ee0d068) = 0
close(24) = 0
write(1, "WilDev/wil0/wlan60-1: applyConfig lhg60-dk RBLHGG-60ad LHG 60G\n", 63) = 63
ioctl(23, SIOCGIFFLAGS, {ifr_name="wil0", ifr_flags=IFF_UP|IFF_BROADCAST|IFF_RUNNING|IFF_MULTICAST}) = 0
ioctl(23, SIOCSIFFLAGS, {ifr_name="wil0", ifr_flags=IFF_BROADCAST|IFF_MULTICAST}) = 0
ioctl(23, SIOCSIFHWADDR, {ifr_name="wil0", ifr_hwaddr={sa_family=ARPHRD_ETHER, sa_data=24:18:1d:98:a4:2e}}) = 0
write(1, "WilDev/wil0/wlan60-1: fw: /pckg/wireless/lib/firmware/wil6210.fw\n", 65) = 65
open("/pckg/wireless/lib/firmware/wil6210-lhg-span2.8.msg", O_RDONLY) = 24
read(24, "M2\354\7\0\250@\0\22\0M2\1\0\0\10t\377\377\377\2\0\0\10t\377\377\377\22\0M2\1\0\0\10\234\377\377\377\2\0\0\10t\377\377\377\22\0M2\1\0\0\10\304\377\377\377\2\0\0\10t\377\377\377\22\0M2\1\0\0\10\354\377\377\377\2\0\0\10t\377\377\377\17\0M2\1\0\0\t\24\2\0\0"..., 4096) = 1100
read(24, "", 4096) = 0
close(24) = 0
ioctl(23, _IOC(_IOC_NONE, 0x89, 0xf8, 0), 0x7ee0d040) = 0
write(1, "WilDev/wil0/wlan60-1: bf: /pckg/wireless/lib/firmware/wil6210-lhg-span2.8.brd\n", 78) = 78
open("/pckg/wireless/lib/firmware/wil6210-lhg-span2.8.brd", O_RDONLY) = 24
read(24, "\220~\2309\10\214\330^B\225j[\210\277\230\211\375K\314\177\344\f\230\306\0170\30\377\221}\331\1\7\276\31%\216\7Y&dP\331I\232\243\31b`\373\231\210G]\231\240\255\305\231\261\0261\331\334J\245\331\341\301Y\213\3\367\34*\216~.\232\24t\273\232S\324PZ\i\340Z^\375~\332f\337\37Z\244"..., 4096) = 3588

@robimarko
I compiled hap ac2 image from your repo and found wirelless to be unusable, receieve sensitivity looks really bad. Did you have time to test wirelless performance.

ps: There is a beta ROS build for arm based mikrotik boards with 4.14.131 kernel. LiMe might work to dump ram with that verson...

Well that is due to generic BDFs being used, ones in the ROS are encrpyted and I dont know how to decrypt them.
No, I did not test wireless performance at all.

Yeah, I already tried v7 ROS but without kernel config that they use I cant produce loadable kernel modules, and they are not keen on publishing them yet.

I have been poking around but I am more and more certain that wireless drivers actually decrypt files.
For example ipq4019_ahb.ko uses an external request_firmware function which I have not yet located and uses that to fetch IPQ4019-otp.bin

So we are really needing a way to debug kernel modules itself, but without kgdb or ftracing I dont know

Actually, that strace output looks great - the part you quoted is generated from the function all the breakpoints are supposed to be in.

Did you verify the sha256sum? It has to be the very same executable of course.

If the sha256sum is correct, just core dump it anyway so we can investigate.

Well that is due to generic BDFs being used, ones in the ROS are encrpyted and I dont know how to decrypt them.

@subixonfire means hAP ac² - it has no .brd files, only wil6210 based boards do.

I guess rbextract isn't included in the branch yet, so no caldata would be loaded - which explains the issues for hAP ac².

Yes checked, It is: a125ed0b31fb28b863ae2806f25fd37dba41474ae0f2baca0f73dce9387fba91
Here is the core dump made with breakpoint after driver was insmoded.

Yeah, I know that he means hAP ac2.
It does not have. brd files, but it has something similar.
It uses BDF aka board files, they are calibration specific to that vendors IPQ40xx based product line or even more specific to each board.
This is provided by QCA or their partners to individual vendors and are shipped in firmware and loaded by drivers, this is the reason for ipq-wifi package and the reason why ath10k firmware contains board-2.bin, it originally only contained BDFs for QCA reference design boards but now also contains other BDFs mostly for OpenWrt supported boards.
In all of QCA wisdom they provide BDFs for IPQ4019 for all with the BMI ID (16 for 2.4GHz and 17 for 5GHz), that is why qcom,ath10k-calibration-variant property was introduced to be able to differentiate between BDFs.

Each device also has traditional ART calibration shipped with it and its a difference between BDF and that specific device plus some other things like to indicate specific features like band steering or to include MAC address.
BMI ID is read from that ART calibration and without ART calibration ath10k will fail to probe and register the radio.
We dont see any BMI/BDF related error messages because board-2.bin included in OpenWrt has BMI IDs 16 and 17, but they are only for QCA reference boards.
And using wrong BDFs can cause low performance at best and unstable/crashy radios at worst.

Mikrotik ships BDFs inside one binary, its IPQ4019-otp.bin.
Driver has some clues, but its not too clear how they uncompress them.
Altough, it looks to be compressed somehow again as usual BDF size is 12064 and MikroTik shipped OTP file is only 4.6k

They have a SGMT magic which points to this:


It looks like its LZ compressed and they are feeding it directly into radio with bmi_transfer_lz function.
ath10k has some mention of LZ also, but I have not figured out how to use it

rbextract and hotplug scripts are in that same branch and like I said, without caldata being extracted ath10k would not even register the radio.

I hope this clears some stuff up

Log for WLAN stuff only:

Thanks for the detailed explanation. I'll dive into it later.

As for wireless. Something is off - that core file seems to be for a different wireless binary than the one with sha256sum a125.. - if you figure it out, let me know, that would help.

Anyhow, I think I found the offsets for whatever-you're-running:

0x000a6ab4:	bl	0x110f0
0x000a6b54:	bl	0x108d4
0x000a6b78:	bl	0x2efcc
0x000a6b7c:	ldrb	r3, [r4, #176]	; 0xb0

Those should hit..

Hm, I gotta check to see if I uploaded the correct file.
I did check hash and it matched one posted by you, only explanation is that I uploaded a old core dump.

@ius Here are core dumps for those offsets.
https://drive.google.com/drive/folders/1rteagdLDrQKPUzB_t66jMslIrI3WCM5L?usp=sharing

I have been looking at available info IPQ4019-otp.bin and it looks like its LZ77 compressed, but it also has some metadata in front.
For some reason upstream ath10k which actually support that kind of stuff wont load board.bin altough it recognizes the file format but it fails on uploading over CE on IPQ4019 radios.
This shell script pretty much describes how to make one of those compressed files, but not how to uncompress them.

mikrotik_brd_decrypt.py <input> <output>

Note: LFSR seed ('state') seems to differ between firmware versions. So this only works for the firmware you have right now.

I checked 6.44.x and 6.45.x, neither seems to match what you're running. Please check again.

25ceb924e3f0ad116bde445abb3e11af8b67f1842ad5ba56ed3e778ba0d4017b  wil6210-lhg-span2.8.brd
e8521e946aa78a5861cdb37b1b6fd030fce12fd80e4f9afe663bab9f025de576  dec.brd

Obviously can't test it, but it looks fine to me:

$ xxd dec.brd 
00000000: 0600 0000 3400 0000 3031 3236 0000 0000  ....4...0126....
00000010: 93aa 14d4 0100 0000 040e 0000 0000 0000  ................
1 Like

You were right, for some reason upgrade form 6.43.12 to 6.45.6 a couple of days ago failed.
Can you explain how to get LFSR seed for a specific firmware version?

I checked now and SHA256 sum in 6.45.6 I am now running is:

# sha256sum wireless 
a125ed0b31fb28b863ae2806f25fd37dba41474ae0f2baca0f73dce9387fba91  wireless

I tried used the board file and firmware generated by your script.
Headers are good, but it appears that checksum is not.
Errors:

wil_fw_verify: ERR[ FW ]checksum mismatch: calculated for 561160 bytes 0x23c6c79b != 0x25344813
ERR[ FW ]checksum mismatch: calculated for 3588 bytes 0x6b37878a != 0xd414aa93

If I disable returning an error on checksum board file will load but firmware wont.
So something is still off.

[  531.421411] wil6210 0000:01:00.0 wlan0: wil_reset: Use firmware <wil6210.fw> + board <wil6210.brd>
[  531.433344] wil6210 0000:01:00.0 wlan0: wil_fw_verify: ERR[ FW ]checksum mismatch: calculated for 561160 bytes 0x23c6c79b != 0x25344813
[  531.452333] wil6210 0000:01:00.0 wlan0: wil_fw_handle_record: ERR[ FW ]unknown record type: 100
[  531.452503] wil6210 0000:01:00.0 wlan0: wil_request_firmware: ERR[ FW ]Loading <wil6210.fw> failed, rc -22

Here is how the driver checks it:
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/drivers/net/wireless/ath/wil6210/fw_inc.c?h=v5.3.2#n105

Any clues on this?
Look like some minor tuning

For 6.45.6: state = 0x58cfa159

For that version the output for wil6210-lhg-span2.8.brd looks okay'ish, but the wil6210 header 0126 is no longer present. Crypto should be exactly the same though (in fact, I used this version's binary to recover the algorithm which works for 6.43.12). Beats me.


Easiest way to recover the seed from the wireless binary is by using IDA or Ghidra, but it's not trivial. It's located in the crypto function, i.e. at offset 0x2f834 in 6.45.6. The only function referencing it has plenty of string references to be able to easily locate it in other versions...

Probably makes more sense to recover the seed based on some known plaintext in one of the .brd files it encrypts, as the state is only 32 bits.

1 Like

Thanks,
header missing is an issue as its really important for wil6210.
It contains a lot of important stuff, it even contains the CRC32 to which driver compares generated one.
Something is missing, but I believe that its extremely hard to reverse engineer solely based on decompiled binary.
I have been digging at v7-beta2 but Ghidra wont decompile a part of the binary where seed is

That weirdly does not also work under 6.46b44.
It also has seed in a similar function but it also generates unsuable files.
It also does not work in 6.44.2, well this is really weird.

Well, the seed is unchanged (0x58cfa159) for 6.46, so I assume it still decrypts fine.

Header missing is odd, but the decryption seems fine. I'm inclined to believe their driver just doesn't require it - although I haven't looked at it at all.

The CRC mismatch for the (old) version with header should be investigated further, though.

I'll need to find some time to look at hAP ac² first though..

Hm, you may actually be right.
In these cases without header, its 3516 B while with header its 3588 B which matches the upstream example.
Well, I gues we can easily calculate the CRC32 and use upstream header as template.
Its all on fixed sizes.

For now, I am using one from 6.43.12 with corrected CRC32 in the header, I dont think that their driver even checks the header.
Who knows what modifications they have made

1 Like

I tried copying the 72 first bytes from a "vanilla" brd and corrected the CRC and this is the error I got. Any ideas? We are trying to get the brd from a wAP 60Gx3 AP (https://mikrotik.com/product/wap_60gx3_ap) and at the moment we are successful. The brd file is from 6.46.4 and I can confirm that the seed is 0x58cfa159 but the headers are missing.

[194704.810380] wil6210 0000:01:00.0 wlan0: wil_get_bl_info: Boot Loader struct v2: MAC = b8:69:f4:d5:7a:22 RF = 0x0000 (status 0x0000) bband = 0x00000000
[194704.810788] wil6210 0000:01:00.0 wlan0: wil_get_bl_info: Boot Loader build 255.255.0.7253
[194704.823628] wil6210 0000:01:00.0 wlan0: wil_set_oob_mode: oob_mode to 0
[194704.831237] wil6210 0000:01:00.0 wlan0: wil_reset: Use firmware <wil6210.fw> + board <wil6210.brd>
[194704.871036] wil6210 0000:01:00.0 wlan0: wil_fw_verify: ERR[ FW ]checksum mismatch: calculated for 3588 bytes 0xd7247813 != 0xeb5c4046
[195145.374374] wil6210 0000:01:00.0 wlan0: _wil6210_disconnect: bssid= (null), reason=3, ev-
[195145.420367] wil6210 0000:01:00.0 wlan0: wil_get_bl_info: Boot Loader struct v2: MAC = b8:69:f4:d5:7a:22 RF = 0x0000 (status 0x0000) bband = 0x00000000
[195145.420780] wil6210 0000:01:00.0 wlan0: wil_get_bl_info: Boot Loader build 255.255.0.7253
[195145.433512] wil6210 0000:01:00.0 wlan0: wil_set_oob_mode: oob_mode to 0
[195145.441229] wil6210 0000:01:00.0 wlan0: wil_reset: Use firmware <wil6210.fw> + board <wil6210.brd>
[195145.483901] wil6210 0000:01:00.0 wlan0: wil6210_irq_misc: Firmware error detected, assert codes FW 0x00001368, UCODE 0x00000000
[195145.769436] wil6210 0000:01:00.0 wlan0: wil_fw_core_dump: fw core dumped, size 823296 bytes
[195145.769494] wil6210 0000:01:00.0 wlan0: wil_notify_fw_error: Notify about firmware error
[195145.777131] wil6210 0000:01:00.0 wlan0: wil_fw_error_worker: No recovery - interface is down
[195147.530136] wil6210 0000:01:00.0 wlan0: wil_wait_for_fw_ready: Firmware not ready