Here is a small Perl script that will download the Wiki page and strip everything but the data: https://gist.github.com/Per9/3b6831866ad1fe8484f7dc5f59056d7c
Maybe you can convert that into the format you need?
@Per thanks. But there are some issues:
This cannot be reasonable done on the client side in the browser
- it's still 1.4MB data
- takes a lot of time to generate (>10s for me)
- it's not valid csv data. It contains additional semicolons in the data fields (e.g. &)
However, I can use it to generate a data set in a cronjob. Try to fix the $amp; etc. and serve it on the host the firmware selector. It's a progress. 
I have updated it to decode HTML entities. Yes, the intention was for you to use it with a cron job or similar.
The quality of the resulting relevant data is rather bad. So far I there no good way to get model name to sysupgrade/factory images file name mappings.
If someone has a better idea...
Bad in what way? Is the extraction flawed, or is the data in the page missing or bad? Can you give some examples?
Very well done. Something like this should be on the main OpenWrt website. Please add the WRT32X. Should be generally the same build as WRT3200ACM.
@phinn I have added the wrt32x: firmware-selector, feel free to contribute more entries via pull request.
@Per I have to filter the output using perl and other scripts + heuristics to extract the data. It's painful to do and the resulting data is incomplete. The links to the image might also point to older OpenWrt versions. Since this has to be done on the server side, I would need to completely rewrite the project. I simply do not have the resources and motivation to do so. Maybe somebody else can to step in?
1 Like
Btw. this would be a perfect project for a student in the context of GSoC...
I created a mapping some time ago, if you tell me the required parameters (like resulting image, profile, model, etc) I can automatically recreate it on every new release.
hi @aparcar,
I do not quite understand the list. There are two columns, each with a list of strings (often only one).
Can you explain what the columns mean? Maybe by example?
@mwarning sorry this might be confusing: So build all images and now have a list containing target, profile, model and resulting sysupgrade. To give you and idea, please see here.
The list is not perfect yet, I'll clean it up a bit. The random letters in the sysupgrade filenames are hashes of the resulting manifest, to distinguish between images. However I'll remove that for convenience.
Hope that helps
Here is a second version containing the cleaned filename and only openwrt/18.06.2
The format is
"target": "",
"profile": "",
"model": "",
"sysupgrade": ""
A link to the factory image is currently not detected by the server, however this would be possible if desired...
PS: PostgreSQL is fun select json_agg(t) from ( select distinct on (target, profile, sysupgrade) target, profile, model, substring(sysupgrade, 0, 17) || substring(sysupgrade, 33) as sysupgrade from images where sysupgrade != '' and version = '18.06.2' ) as t \g mapping-v2.json
That looks very useful. I can use model and target+sysupgrade (to create the path to the image file).
A few notes:
- sysupgrade seems to point to factory, sysupgrade or rootfs images etc.
- a vendor field would be nice (the first word of the model field is a suboptimal heuristic)
- a hardware revision would be nice as well (extracting it from the model field is suboptimal as well)
From my point of view, a good format for an entry might be:
{"vendor": "TP-Link", "model": "TL-WR840N", "revision": "v4", "target":"ramips/mt76x8", "profile":"tl-wr840n-v4" "images": ["openwrt-18.06.2-ramips-mt76x8-tl-wr840n-v4-squashfs-sysupgrade.bin", "openwrt-18.06.2-ramips-mt76x8-tl-wr840n-v4-squashfs-factory.bin"]}
(I do not know if profile is needed by anyone, the types of images can be distinguished by filename)
@aprcar: Where/How do you generate your data?
There was an idea to embed a vendor and revision field in the Makefiles of openwrt and generate a json file per target.
Firmware refers to software, bootloader , data etc that is built-in to a device like a router or printer. And not intended to be modified by the user.
The user doesn't normally change the firmware unless the manufacturer provides an update.
Software is any application it can be included in the firmware or added later.
Drivers are special software used by the operating system to control over internal or external devices.
Openwrt is effectively a custom manufacturer for your device that you can use to replace firmware of the original manufacturer.
Probably more to it but hard to cover everything in a comment.
You could think of the firmware as the copy of an entire hard drive as a file (boot loader + operating system + software).
1 Like
Since OpenWrt now has the ability to create json files as output, I have created yet another firmware selector based on auto-completion:
Demo: https://mwarning.de/yet_another_firmware_selector/
Sources: https://github.com/mwarning/yet_another_firmware_selector
Maybe someone can generate all json files and make them available so I can feed them to the selector demo?
Disclaimer: the design is a copy of https://sudhanshu16.github.io/openwrt-firmware-selector/, but the code is too heavy for my taste (I do not know node very well).
1 Like
I am blown away... This is a thing of beauty.
But... That doesn't prevent me from offering a couple comments/requests:
- Can this also work with the stable release builds (currently 19.07.0)? If so, that could be the default.
- Could you alphabetize the names in the suggestion list (I almost snagged the wrong one from the list below.)
And finally, do you think this is good enough to make an official part of the OpenWrt site? Many thanks!
@richb-hanover-priv I can provide stable release builds by duplicating the json data for openwrt-19.07, since the feature to create json files is not there yet.
Sorting the results should be no problem. EDIT: done
One of my long-term desires is to make OpenWrt easier to use. The Firmware Wizard would go a long way, eliminating the need to refer newcomers to this scary page.
(I tweaked the script that generates that page - it was even worse before - but it still requires people to know way too much random stuff about their router: What's a target? Or should I look in packages? Do I want generic? Or nand? And how tiny is "tiny"?)
My hope is that we can evolve an official OpenWrt page to provide this capability. We wouldn't stop creating the "scary pages", but the Firmware Wizard could provide all that information, keyed on information the newcomer does know (like the name & model of their router).
My question to you: What would you need from the build process to be able to do this for full releases as well as the nightly snapshots? Thanks.
1 Like