RFC: rendering Table of Hardware via yaml files

Hey all,

I'm a LineageOS user and therefore sometimes end up in their Wiki. The objective is somewhat similar to OpenWrt, they offer alternative (linux) firmware for all kinds of devices. Instead of using a MediaWiki all device specific pages are rendered based on a .yaml file.

Installation methods are inserted via templates while also allowing custom notes.

Currently developers add device information only in the commit messages, they could be advised to create a .yaml file in a (to be created) repo like devices.git.

What do you think of moving to this system? It would introduce some changes, however could greatly uni- and simplify the wiki.

I'd be happy to work on this as I see it as a big usability improvement.

2 Likes

A proof of concept is here with the sources here

The scraper.py script was used to create the yaml files, most of the layout is adapted from lineage and just for demo.

2 Likes

I added a search function and download buttons

Looks good. This approach certainly adds value, but creating multiple sources of information can be a headache.

Maybe the yaml files can be used to create a header for each site in the openwrt wiki and to populate the table of hardware in the wiki? The rest of the wiki site can be populated by the wiki users with all the information that does not have a place in the yaml files (howtos, serial port info, pictures), as usual.

@aparcar I think this yaml info can be combined with the info from the json files of the build system (GSoC project) to create a nice firmware selector/wizard.
Then the build system would also need only minimal data.

The image above is rendered using the following metadata:

availability: Discontinued 2015
bootloader: U-Boot
brand: TP-Link
comment_installation: Rename installation image to wdr3500v1_tp_recovery.bin before
  flashing.
cpu: Atheros AR9344
cpu_cores: 1
cpu_mhz: 560
detachable_antennas: '2'
device_id: tp-link_tl-wdr3500_v1
device_page: tl-wdr3500
device_type: WiFi Router
ethernet_100m_ports: 5
fccid: https://fcc.io/TE7/WDR3500
firmware_openwrt_install_url: http://downloads.openwrt.org/releases/18.06.2/targets/ar71xx/generic/openwrt-18.06.2-ar71xx-generic-tl-wdr3500-v1-squashfs-factory.bin
firmware_openwrt_snapshot_install_url: http://downloads.openwrt.org/snapshots/targets/ar71xx/generic/lede-ar71xx-generic-tl-wdr3500-v1-squashfs-factory.bin
firmware_openwrt_snapshot_upgrade_url: http://downloads.openwrt.org/snapshots/targets/ar71xx/generic/lede-ar71xx-generic-tl-wdr3500-v1-squashfs-sysupgrade.bin
firmware_openwrt_upgrade_url: http://downloads.openwrt.org/releases/18.06.2/targets/ar71xx/generic/openwrt-18.06.2-ar71xx-generic-tl-wdr3500-v1-squashfs-sysupgrade.bin
flash_mb: 8
forum_search: TL-WDR3500
git_search: TL-WDR3500
gluon_support: true
malformed: {}
model: TL-WDR3500
package_architecture: mips_24kc
ram_mb: 128
recovery_method_s: U-Boot TFTP recovery
serial: true
subtarget: generic
supported_current_rel: 18.06.2
supported_since_commit: https://git.lede-project.org/?p=source.git;a=commit;h=492cbcdd6bb41392e3e242c8435855477dd37dc6
supported_since_rel: '14.07'
switch: Atheros AR9344
target: ar71xx
usb_ports: 1x 2.0
version: v1
vlan: true
wikidevi_url: https://wikidevi.com/wiki/TP-LINK_TL-WDR3500
wlan_24ghz:
    - b
    - g
    - n
wlan_50ghz:
    - a
    - n
wlan_driver: ath9k
tftp_image: wdr3500v1_tp_recovery.bin
tftp_ip_local: 192.168.0.66
tftp_ip_device: 192.168.0.86
tftp_image: wdr3500v1_tp_recovery.bin
image: tp-link_tl-wdr3500_v1.jpg
switch_ports:
    0: CPU
    4: LAN 1
    3: LAN 2
    2: LAN 3
1: LAN 4

Jep, thats what I though of. I'm trying to build a system which is then fed by the buildroot regarding image names and stuff.

How do you add new / edit existing devices?

@tmomas This is just a lot of ideas being thrown around. I think for the openwrt repository a device would be added as usual. An optional yaml file can be added to a "devices" or to the "openwrt" repository (no idea if it is better to have a distinct repository). Some information from there can be migrated back to the yaml files.

For the wiki, the yaml file would be used to create a wiki site. Under the wiki entry, all kinds of information can be added as usual.

The web interface shown by @aparcar would be separate from the wiki, but contains a generated link to the wiki site of the selected device.

What do you think?

Hi, to avoid confusion, @mwarning response is not entirely in line with my actual idea.

There may be an GSoC project where a student designs a firmware selector and trains the buildroot to output json files containing image information. However, my approach here is to unify the wiki as much as possible. The yaml file above is 98% based on the device page, using a simple parser. The TFTP text is a template, fed with the tftp_* variables. Also the switch_ports-dict results in a always same looking table.

My propose would be to parse all existing information and evaluate what templates are needed, like the TFTP one, but for cases. Each device would have a single file containing only a few variables.

Additionally the GSoC project could result in a way to parse actual download links and firmware image names. This could be integrated on the fly.

Lastly, we could motivate openwrt.git developers to supply a yaml file (based on a template) the moment they propose to add a new device.

Extended the parser script to parse existing images