Toward a good "Flashing LEDE Instructions" page

Update: I removed those "factory" and "sysupgrade" notes, since they wouldn't be true for large classes of devices, and they don't really make the process more reliable.

I think @jow might be referring to the fact you can use the installation ('factory') image to perform a sysupgrade as well on x86. That had me puzzled at first on my x86/64 APU2.

E.g. This is what you'd use for an ext4 rootfs with overlay, afaik:

https://downloads.lede-project.org/snapshots/targets/x86/64/lede-x86-64-combined-ext4.img.gz

Yes, I can see the confusion. I removed the comments (from the Flashing Instructions page) about 'factory' and 'sysupgrade' in the file name.

People rely on the Device Page, which ultimately gets its information from the 'dataentry' page that a person created when adding the device. Presumably, the original author enters the URLs correctly.

In cases where the names of factory/sysupgrade don't match a pattern (or, if they're the same in your case), the Device Page would be a good place to note that... (That is vastly simpler than trying to come up with a complicated algorithm for when a device has/doesn't have a particular string...)

I added a new field "Installation method" to the dataentries, with possible values "standardflashinginstructions|OEM GUI", "flashwithssh|CLI", "flashwithhardwaremod|Hardware mod"

with

< pagename for instructions > | < title to be displayed >

The possible values can be changed as we like, e.g. as proposed "tp-link_interface_2017-2020", just let me know and I'll add this to the dropdown.

EDIT: pagenames should contain at least a part of the title to be displayed, since filtering for installation methods in the datatables works per pagename, not per title.

[This note may belong on the "Dataentry" topic... If so, let's move the discussion there.]

I am not sure of the value of this "Installation method" field.

First off, I don't know how we would populate it. It wouldn't make sense for us to review all the Device Pages from OpenWrt: despite the fact that it would be a lot of work, we'd simply propagate any (mis)information contained there (because we don't have personal experience with all devices).

Second, what will the reader do differently based on the setting of this field? Would they choose not to use the device?

So I would choose not to retain this field. Thanks.

Following up on my post on packages yesterday, and assuming these still, under LEDE, may change frequently related to the nightly builds, I think there is value to add something to the "Standard Flashing Instructions" on this topic. This could be a note under "Preliminary Steps" - " Special Considerations for Trunk Builds" and discuss the dynamic aspect of nightly builds, saving the relevant packages locally and how to point LEDE to this repository.

As a newbie, I was burnt on this when I first started with OpenWrt and I believe there have been more than one post on it in the forums.

Is your concern that: These days, LEDE development is in flux. We should warn people that the packages available on a particular day might not work with an image retrieved the day before.

If so, perhaps the site could give a couple bits of guidance:

  • In late 2016, the LEDE software is changing rapidly, getting better day by day. But if you are looking for stable, bug-free router firmware, LEDE may not be a good choice.

  • If you do wish to use LEDE, then you should plan to download the firmware image and then install/update packages in the same 24 hour period, to be sure that they are always compatible.

Would this be good advice?

Is your concern that: These days, LEDE development is in flux. We should warn people that the packages available on a particular day might not work with an image retrieved the day before.

(Not sure how to quote you)

Development is always in flux, but yes I am suggesting we be clear, focused on new users, that the repository changes frequently (generally every 1-3 days) and that it's important, if they want to add functionality, that they also save the matching (400MB or so) packages at the same time. I don;t think it can hurt to make a statement about trunk, even if already made elsewhere, that trunk may be more buggy and does not include LuCi, etc.

We do not have a "Stable" release, but as I read the instructions, I feel that they are focused on a stable release. For new devices, the latest release may be trunk (there are a few DD Trunk in the OpenWrt TOH). It may be 6-12 months until they are in the next stable release.

NOTE - Currently for all platforms there is (For new devices there may be) no stable release. We run buildbots frequently to update trunk with the latest firmware and packages. Many of these packages need to match the appropriate level of firmware. If the latest release for your device is "trunk" and you want to enhance the capabilities of your device with packages, you should download both the target and instruction set packages to your local environment when you download your firmware. See this note on how to save the files and options on how to load them to your device. See this note for the differences between a stable release and trunk release.

Notes not included, but I expect include wget use, how to load individual packages to the deicev (WinSCP or FTP, etc) and how to update the pointers to the available repository on ones lan or attached USB device, etc.

As the saying goes: Make it idiot proof and someone will build a better idiot.

OpenWrt wiki has an infobox ready-to-be-included in any page for this purpose: https://wiki.openwrt.org/meta/infobox/attention_trunk

I think that about sums it up, but are you suggesting we link to OpenWrt or borrow the "open source content"?

Did not know these existed.

No need to make it idiot proof. “LEDE is primarily intended for networking enthusiasts, wireless communities, embedded Linux developers and advanced users.” https://wiki.lede-project.org/wiki/start#audience

Advanced users should know the basics. If not, they are not advanced users.

A very good point, but they may sneak in under the "enthusiasts" category.

Should we change that to networking professionals?

It's almost as if we should be more explicit about the criteria (“LEDE is primarily intended for networking enthusiasts, wireless communities, embedded Linux developers and advanced users.”) and say that something like (slightly tongue in cheek):

  • LEDE is very cool
  • But... LEDE is changing fast, and may not be stable
  • Do not use LEDE as your primary router until we put out our first stable release (in ??? 2017)
  • But if you want to help us move the state of the art ahead in router firmware, please install and test our builds, and report your successes and problems

[quote="richb-hanover, post:30, topic:52, full:true"]say that something like (slightly tongue in cheek):[/quote]Rephrased a bit in two sentences below:

LEDE is great but is changing fast, we think it is not stable enough to be used on your primary router.
We are working hard towards a stable release though, if you want to help us feel free to install and test our builds, report issues and send fixes.

I think we are confusing 2 different things.
The current state of LEDE dev. There is NO released version.
The target audience of LEDE long term.
Both are relevant.

While quality is always an issue, but there is as I have suggested, a big difference for the user to manage trunk and a released version with a stable repository, as I learned last night. After downloading an image and respective packages, it took an hour or more to install Luci. I had to find the files and move them to the router, then install. My choice of HooToo TM-02 had only a LAN Ethernet and of course wireless was not configured. Even so it's not clear (to me) with less than a working USB drive for the device how to path to the "LAN" based repository. (read the OpenWrt wiki). I suppose if one installs immediately after downloading, then the packages are still valid. If they install at least USB then the repository can be configured on the USB drive, assuming they have a device with USB.

I like this. I think this belongs on the Quick Start page, though, so I put (slightly reworded) language there. Thanks!

You are right - there are two things to consider:

  • Long term, what do we want the LEDE download-and-install experience to be?
  • What do people have to do right now, in Nov 2016, to successfully install and test?

The doc's that I've been writing envision the long-term stable release. I want to drive toward this as a goal. We should strive to make LEDE as simple as this, but the instructions I've written may be premature.

We need to come up with guidance for people to use now. I am frankly not up to speed enough with the procedure to offer advice. Do we tell people:

  • download and install and opkg update/install everything in the same 15 minute period?
  • Download the image and all (400 mbytes?) of packages, save them to a local file system, then pick and choose?
  • Something else...?

We should write out the steps for "Installing LEDE in late 2016" (that gives all the gory details) and add a link to the end of the warning at the top of the Quick Start page.

Maybe it's more like:
1 - Installing from Stable
2 - Installing from Trunk

There always will be a trunk and the same issues, irrespective of how new LEDE is.

I expect 2 will mostly expand on 1, but for example I just noticed that in QSG, step 7 we say install Luci, which (I expect) will already be included with Stable Release.

The big issue in my mind with trunk is package management. I think we should suggest that a user "plan" to download firmware and packages at the point they plan to do the install and suggest they immediately install Luci and USB (May need a page for USB, i do not see it in the user guide). Storing the packages on a USB drive is probably the easiest way to address enhancing a particular build for an ongoing period.

I'm not smart enough to improve on the content of the OpenWrt WIKI on the local repositories, but think we will need this or similar. To me "local" means a device on the router (USB), but would hope that there are ways that we can point to a network device (including Windows), ftp site, or users webserver (local or remote). I tried to put the files on Dropbox, but wget was not set up for https.

Maybe worth reminding ourselves that some users will need truck as it's the only available firmware for their new router. To them it does not matter much if it's testing SW, it's the only SW. Almost all the GLi routers are supported in LEDE trunk, but none in 15.05.1. These users may not want to upgrade if the firmware servers their needs, which it hopefully will for core functions.

I had an issue with my newly flashed device here. Appears we may need to add a note about clearing the browser cache after installing Luci.

[quote="RangerZ, post:35, topic:52, full:true"]The big issue in my mind with trunk is package management. I think we should suggest that a user "plan" to download firmware and packages at the point they plan to do the install and suggest they immediately install Luci and USB. Storing the packages on a USB drive is probably the easiest way to address enhancing a particular build for an ongoing period.[/quote]Agree on general idea, adding my OpenWRT veteran experiences on it.

The best way to use devices kept on trunk is to integrate all packages in a flashable firmware image (using Image Builder, we currently don't have a tutorial in the wiki for this, must be imported from OpenWRT wiki) and do sysupgrades relatively often.

You can cram a surprising amount of things (for a router) in a 8 MiB firmware image as the Image Builder places everything in the highly compressed squashfs partition, doing extroot (expanding firmware in a USB/Sata/SDcard/whatever) isn't really required unless the device has 4 MiB of internal space or you are using it as a miniserver.

[quote](May need a page for USB, i do not see it in the user guide)[/quote]The page is here, but the fact you missed it probably means its title should be changed.
Currently the title in the user guide list is "Extroot configuration (expand your firmware space in a storage device)".
Feel free to come up with a better title.

[quote]but wget was not set up for https.[/quote]jow said that the onboard "wget" is actually uclient-fetch and that it should be able to use SSL if you install SSL libraries https://github.com/lede-project/source/pull/463#issuecomment-257545072 (and a ls -l /bin also confirms this, the "wget" tool is a symlink to uclient-fetch, not to busybox.)

I've just tested this on my guinea pig router, with PolarSSL library (not installed by default), but wget fails with "Invalid SSL certificate", because there is no certificate database in the default firmware (ca-certificates is the package name for that I think).

root@lede:/# wget -O /tmp/page https://wiki.openwrt.org/doc/howto/netfilter
Downloading 'https://wiki.openwrt.org/doc/howto/netfilter'
Connecting to 81.0.124.216:443
Connection error: Invalid SSL certificate

You can override the certificate check by giving it an option to not check certificates "--no-check-certificate", it is of course not a good thing to not check certificates, but if you need to download a package from dropbox or something like that in a pinch it will work fine from default LEDE firmware.

root@lede:/# wget --no-check-certificate -O /tmp/page https://wiki.openwrt.org/doc/howto/netfilter
Downloading 'https://wiki.openwrt.org/doc/howto/netfilter'
Connecting to 81.0.124.216:443
Writing to '/tmp/page'
/tmp/page            67344  --:--:-- ETA
Download completed (67344 bytes)
1 Like