My solution is a single tool that can handle all the formats generally and has a database for holding model specific metadata. I've made good progress on this but to build a database for the metadata I need copies of original firmwares. This is where I could use some help.
What I need is for someone to update the individual scrapers/spiders from https://github.com/firmadyne/scraper which is written in Python. I don't know Python and I know some people eat, drink and breath Python code. Also, instead of actually just downloading the files, I'd like it to spit out the company name, model and firmware version and firmware URL in CVS format for each and every device and firmware.
This is part of a larger effort to automate firmware analysis because currently devices are being released faster than they can be supported.
If you're interested in helping, please message me.
This has been tried multiple times already, the result was indeed just adding yet another syntax to the mix - instead of 'replacing' anything (remember wajig?). Once you need functionality beyond the mere basics covered by your generic wrapper, you still have to remember to native commands (and that happens quicker than you might think, think dpkg --get-selections or update-alternatives on Debian systems) - aside from the problem that not a single system except of your own will ship with this wrapper by default (and even you will have to remember the native syntax whenever you set up a new system).
As far as OpenWrt is concerned, the python dependency is effectively a killer for this tool anyways. python is not part of the default package set - and it's large, too large for most plastic routers with ~8-16 MB flash (and performance/ RAM usage aren't that inviting either).
While that negates the performance aspects, the "standards proliferation" angle so nicely summarized by xkcd still stands.
Sure, having a common tool would be cool - but different vendors still need very different firmware formats. On the one hand there are the legacy dont_touch_if_it_aint_broken ones[0], on the other hand vendors will always come up with new requirements, so new ones not matching whatever you envision now will emerge. The old UN*X mantra or one tool for the job imho applies, the needs for these devices are just that diversive and continue to diverge.
--
[0] Yes, their requirements might be cast in stone, but the finer details will have been forgotten with the devices going out of fashion and ending up on various landfills. Verifying that the wrapper actually still works won't be possible for a lot of them. Yes, that is also an issue for the existing packaging helpers (when faced with newer libraries, compilers, etc.), but there you have at least better chances that you might get it right or are able to bisect the issue.
I'm fully aware of this issue. I've implemented a dozen different formats already. The testing on real hardware is not as big an issue as it once was because it can be tested via emulation which is another part of the automated firmware analysis.
You're poo-pooing my project before even understanding the purpose which doesn't help anyone. Please read the full post before spouting off again.
