Sure, I it would be great as I should have plenty of time after this week.
So I can take one since you have extras.
Hm, the thing is that I doubt that maintainers would love a remake of ar71xx mess and amount of maintenance it required due to custom code.
It would be best to avoid additional code if not necessary.
You can easily set MAC in 02_network with mtd_get_mac_binary function like this:
@idahorazor Take a look at this:
You can find LZO-RLE decompressor in C already.
I am trying to figure in what kind of encoding/compression are files in /bndl/wireless/lib/firmware/ encoded/compressed as they all appear to use the same protection used by MikroTik
@idahorazor I am facing a weird issue.
For whatever reason your Python script wont parse my caldata.
I always get:
ERD chunk_id=0x1 size=0x2f42
unexpected ERD size (multiple chunks?)
Traceback (most recent call last):
File "./mikrotik_radio_data.py", line 238, in <module>
data = radio_data_rle_decompress(data[:-1])
File "./mikrotik_radio_data.py", line 204, in radio_data_rle_decompress
out += bytes([data[i]]) * x
IndexError: index out of range
It was my script. The 'unexpected ERD size' is a fair warning.
If you can upload the partition data you're trying to parse somewhere, I might be able to take a look.
Sorry, I was unaware you are the author.
Here is the full dump and hard_config
https://drive.google.com/drive/folders/1YwUe86HWSLIxAOb85YMRKxltAmIyVLBv?usp=sharing
Ah, having two different dumps helps. Found & fixed two issues with the script - I've updated it.
It now yields the same amount of bytes (65536) as it does for the other dump, so I hope it's now correct.
Thanks, it produces an output now.
Will test whether ath10k likes it.
So good news, it actually works.
Hacked some new functions into hotplug to do the magic on boot and radios are probed and registered.
@ius
Now to debbuging the rbextractor C based app to get it to properly decode caldata as its output is different than your python script.
Any input on LZOR or RLE decoding would be awesome as I am not really good with that.
Source is here:
Anybody good with C willing to help finish the rbextract?
I am struggling to extract the caldata, I can easily find the LZOR tag but I have no idea how to dig the ERD out of it and then decompress.
Hmm, for some reason I missed all new messages here for the last month!
@robimarko there should be a fairly clean DTS file here: https://github.com/mmaker/openwrt/tree/device/hAP-ac²
It's based on one of your tree for the LHGG-60ad.
I will try to have a look at your rbextract. Do you have a way to quickly test it?
DTS and most stuff is already done(Saw that branch after I manually done it already)
Yeah, rbextract and my whole hap ac2 branch is here:
You can simply compile it on x86 and test on hap ac2 hard_config dump as they are both little endian.
Simply clone and make on source.
That is how I am developing it.
I am currently stuck on caldata as this is quite different than old ar71xx based RB.
As discussed on the github thread https://github.com/openwrt/openwrt/pull/2417 I ported the extraction method to C and it gives the same result as the python script, thanks a lot @ius !
The code is here: https://github.com/zorun/openwrt/commit/2e4099910c7784a588f17375a9e073b9610a652a
The only strange thing is that lzo1x_decompress_safe returns a LZO_E_INPUT_NOT_CONSUMED error, which means that the input has too much data.
@ius I don't think it's a big problem, but do you have any idea where that could come from?
Awesome, gonna try it tommorow.
Well it may not be an error:
LZO_E_INPUT_NOT_CONSUMED
The end of the compressed block has been detected before all
bytes in the compressed block have been used.
This may actually not be an error (if 'src_len' is too large).
Does this help with RB3011 at all? I feel like this thread heavily diverged from the original topic of porting OpenWrt to RB3011 with unrelated discussion.
Well this thread has in the end really become a mix of all things MikroTik porting related.
I dont mind it, but maybe changing the title would be smart
@bjonglez
There is something weird happening, when using rbextract on desktop identical files are generated.
But when using it on hAP completely different file is generated.
Weirdly under GCC 9.2.1 on my Fedora desktop with -Wall which would show all warnings I get no warnings but under OpenWrt toolchain I get some warnings:
rbextract.c: In function '__rb_get_wlan_data':
rbextract.c:309:23: warning: passing argument 4 of 'lzo1x_decompress_safe' from incompatible pointer type [-Wincompatible-pointer-types]
buf_lzo_out, &lzo_out_len, NULL);
^
In file included from rbextract.c:25:0:
minilzo.h:94:1: note: expected 'lzo_uint * {aka long unsigned int *}' but argument is of type 'size_t * {aka unsigned int *}'
lzo1x_decompress_safe ( const lzo_bytep src, lzo_uint src_len,
^~~~~~~~~~~~~~~~~~~~~
rbextract.c: In function '__rb_get_wlan_data.isra.1':
rbextract.c:267:11: warning: 'buf_rle_out' may be used uninitialized in this function [-Wmaybe-uninitialized]
uint8_t *buf_rle_out;
This smells like either CCFLAGS or LDFLAGS issue.
@bjonglez @ius So, I was thinking that maybe minilzo is the issue so I moved to using liblzo and dynamicly linked that in.
Also moved to CMake because linking was not working with traditional Makefile.
Now it again works on desktop but on the router different file is generated.
This ought to be RLE somehow producing different file because LZO output from router and desktop is the same.
EDIT: Its definitively RLE for whatever reason not reading the same thing in the same offset.
I again dumped LZO output and they match.
For example on desktop:
srclen: 12096
dstlen: 65536
srcpos: 1
RLE count: -4
srcpos: 6
RLE count: 2
RLE c: 1
srcpos: 8
RLE count: -6
srcpos: 15
RLE count: 2
RLE c: 0
srcpos: 17
RLE count: -4
And on the router:
srclen: 12096
dstlen: 65536
srcpos: 1
RLE count: 252
RLE c: 32
srcpos: 3
RLE count: 47
RLE c: 205
srcpos: 5
RLE count: 209
RLE c: 2
srcpos: 7
RLE count: 1
RLE c: 250
srcpos: 9
RLE count: 116
RLE c: 77
srcpos: 11
RLE count: 40
RLE c: 237
srcpos: 13
RLE count: 163
RLE c: 246
srcpos: 15
RLE count: 2
RLE c: 0
srcpos: 17
RLE count: 252
RLE c: 32
srcpos: 19
RLE count: 0
Desktop one goes for the full input buffer while on the router above is the whole output.
It does not even go through the whole buffer.
Upon further debugging it exits because input at srcpos 19 equalls to 0 so it exits like designed.
Issue is that I dont understand how could the same code read different things at the same position on 2 different architectures when input buffer from which its reading is equal.
Type mismatch. char is signed on x86, but on ARM it's unsigned. Try making count an explicit signed char.
That makes sense, will give it a go.
And that was actually it.
Awesome catch, I would have never remember a difference like that
Glad you figured that weird thing out!
Is there anything else missing for hap-ac2 support?