Toward a good "Flashing LEDE Instructions" page

Talk about Documentation
21

I added a new field "Installation method" to the dataentries, with possible values "standardflashinginstructions|OEM GUI", "flashwithssh|CLI", "flashwithhardwaremod|Hardware mod"

with

< pagename for instructions > | < title to be displayed >

The possible values can be changed as we like, e.g. as proposed "tp-link_interface_2017-2020", just let me know and I'll add this to the dropdown.

EDIT: pagenames should contain at least a part of the title to be displayed, since filtering for installation methods in the datatables works per pagename, not per title.

22

[This note may belong on the "Dataentry" topic... If so, let's move the discussion there.]

I am not sure of the value of this "Installation method" field.

First off, I don't know how we would populate it. It wouldn't make sense for us to review all the Device Pages from OpenWrt: despite the fact that it would be a lot of work, we'd simply propagate any (mis)information contained there (because we don't have personal experience with all devices).

Second, what will the reader do differently based on the setting of this field? Would they choose not to use the device?

So I would choose not to retain this field. Thanks.

23

Following up on my post on packages yesterday, and assuming these still, under LEDE, may change frequently related to the nightly builds, I think there is value to add something to the "Standard Flashing Instructions" on this topic. This could be a note under "Preliminary Steps" - " Special Considerations for Trunk Builds" and discuss the dynamic aspect of nightly builds, saving the relevant packages locally and how to point LEDE to this repository.

As a newbie, I was burnt on this when I first started with OpenWrt and I believe there have been more than one post on it in the forums.

24

Is your concern that: These days, LEDE development is in flux. We should warn people that the packages available on a particular day might not work with an image retrieved the day before.

If so, perhaps the site could give a couple bits of guidance:

  • In late 2016, the LEDE software is changing rapidly, getting better day by day. But if you are looking for stable, bug-free router firmware, LEDE may not be a good choice.

  • If you do wish to use LEDE, then you should plan to download the firmware image and then install/update packages in the same 24 hour period, to be sure that they are always compatible.

Would this be good advice?

25

Is your concern that: These days, LEDE development is in flux. We should warn people that the packages available on a particular day might not work with an image retrieved the day before.

(Not sure how to quote you)

Development is always in flux, but yes I am suggesting we be clear, focused on new users, that the repository changes frequently (generally every 1-3 days) and that it's important, if they want to add functionality, that they also save the matching (400MB or so) packages at the same time. I don;t think it can hurt to make a statement about trunk, even if already made elsewhere, that trunk may be more buggy and does not include LuCi, etc.

We do not have a "Stable" release, but as I read the instructions, I feel that they are focused on a stable release. For new devices, the latest release may be trunk (there are a few DD Trunk in the OpenWrt TOH). It may be 6-12 months until they are in the next stable release.

NOTE - Currently for all platforms there is (For new devices there may be) no stable release. We run buildbots frequently to update trunk with the latest firmware and packages. Many of these packages need to match the appropriate level of firmware. If the latest release for your device is "trunk" and you want to enhance the capabilities of your device with packages, you should download both the target and instruction set packages to your local environment when you download your firmware. See this note on how to save the files and options on how to load them to your device. See this note for the differences between a stable release and trunk release.

Notes not included, but I expect include wget use, how to load individual packages to the deicev (WinSCP or FTP, etc) and how to update the pointers to the available repository on ones lan or attached USB device, etc.

As the saying goes: Make it idiot proof and someone will build a better idiot.

26

OpenWrt wiki has an infobox ready-to-be-included in any page for this purpose: https://wiki.openwrt.org/meta/infobox/attention_trunk

27

I think that about sums it up, but are you suggesting we link to OpenWrt or borrow the "open source content"?

Did not know these existed.

28

No need to make it idiot proof. “LEDE is primarily intended for networking enthusiasts, wireless communities, embedded Linux developers and advanced users.” https://wiki.lede-project.org/wiki/start#audience

Advanced users should know the basics. If not, they are not advanced users.

29

A very good point, but they may sneak in under the "enthusiasts" category.

Should we change that to networking professionals?

30

It's almost as if we should be more explicit about the criteria (“LEDE is primarily intended for networking enthusiasts, wireless communities, embedded Linux developers and advanced users.”) and say that something like (slightly tongue in cheek):

  • LEDE is very cool
  • But... LEDE is changing fast, and may not be stable
  • Do not use LEDE as your primary router until we put out our first stable release (in ??? 2017)
  • But if you want to help us move the state of the art ahead in router firmware, please install and test our builds, and report your successes and problems
31

[quote="richb-hanover, post:30, topic:52, full:true"]say that something like (slightly tongue in cheek):[/quote]Rephrased a bit in two sentences below:

LEDE is great but is changing fast, we think it is not stable enough to be used on your primary router.
We are working hard towards a stable release though, if you want to help us feel free to install and test our builds, report issues and send fixes.

32

I think we are confusing 2 different things.
The current state of LEDE dev. There is NO released version.
The target audience of LEDE long term.
Both are relevant.

While quality is always an issue, but there is as I have suggested, a big difference for the user to manage trunk and a released version with a stable repository, as I learned last night. After downloading an image and respective packages, it took an hour or more to install Luci. I had to find the files and move them to the router, then install. My choice of HooToo TM-02 had only a LAN Ethernet and of course wireless was not configured. Even so it's not clear (to me) with less than a working USB drive for the device how to path to the "LAN" based repository. (read the OpenWrt wiki). I suppose if one installs immediately after downloading, then the packages are still valid. If they install at least USB then the repository can be configured on the USB drive, assuming they have a device with USB.

33

I like this. I think this belongs on the Quick Start page, though, so I put (slightly reworded) language there. Thanks!

34

You are right - there are two things to consider:

  • Long term, what do we want the LEDE download-and-install experience to be?
  • What do people have to do right now, in Nov 2016, to successfully install and test?

The doc's that I've been writing envision the long-term stable release. I want to drive toward this as a goal. We should strive to make LEDE as simple as this, but the instructions I've written may be premature.

We need to come up with guidance for people to use now. I am frankly not up to speed enough with the procedure to offer advice. Do we tell people:

  • download and install and opkg update/install everything in the same 15 minute period?
  • Download the image and all (400 mbytes?) of packages, save them to a local file system, then pick and choose?
  • Something else...?

We should write out the steps for "Installing LEDE in late 2016" (that gives all the gory details) and add a link to the end of the warning at the top of the Quick Start page.

35

Maybe it's more like:
1 - Installing from Stable
2 - Installing from Trunk

There always will be a trunk and the same issues, irrespective of how new LEDE is.

I expect 2 will mostly expand on 1, but for example I just noticed that in QSG, step 7 we say install Luci, which (I expect) will already be included with Stable Release.

The big issue in my mind with trunk is package management. I think we should suggest that a user "plan" to download firmware and packages at the point they plan to do the install and suggest they immediately install Luci and USB (May need a page for USB, i do not see it in the user guide). Storing the packages on a USB drive is probably the easiest way to address enhancing a particular build for an ongoing period.

I'm not smart enough to improve on the content of the OpenWrt WIKI on the local repositories, but think we will need this or similar. To me "local" means a device on the router (USB), but would hope that there are ways that we can point to a network device (including Windows), ftp site, or users webserver (local or remote). I tried to put the files on Dropbox, but wget was not set up for https.

Maybe worth reminding ourselves that some users will need truck as it's the only available firmware for their new router. To them it does not matter much if it's testing SW, it's the only SW. Almost all the GLi routers are supported in LEDE trunk, but none in 15.05.1. These users may not want to upgrade if the firmware servers their needs, which it hopefully will for core functions.

36

I had an issue with my newly flashed device here. Appears we may need to add a note about clearing the browser cache after installing Luci.

37

[quote="RangerZ, post:35, topic:52, full:true"]The big issue in my mind with trunk is package management. I think we should suggest that a user "plan" to download firmware and packages at the point they plan to do the install and suggest they immediately install Luci and USB. Storing the packages on a USB drive is probably the easiest way to address enhancing a particular build for an ongoing period.[/quote]Agree on general idea, adding my OpenWRT veteran experiences on it.

The best way to use devices kept on trunk is to integrate all packages in a flashable firmware image (using Image Builder, we currently don't have a tutorial in the wiki for this, must be imported from OpenWRT wiki) and do sysupgrades relatively often.

You can cram a surprising amount of things (for a router) in a 8 MiB firmware image as the Image Builder places everything in the highly compressed squashfs partition, doing extroot (expanding firmware in a USB/Sata/SDcard/whatever) isn't really required unless the device has 4 MiB of internal space or you are using it as a miniserver.

[quote](May need a page for USB, i do not see it in the user guide)[/quote]The page is here, but the fact you missed it probably means its title should be changed.
Currently the title in the user guide list is "Extroot configuration (expand your firmware space in a storage device)".
Feel free to come up with a better title.

[quote]but wget was not set up for https.[/quote]jow said that the onboard "wget" is actually uclient-fetch and that it should be able to use SSL if you install SSL libraries https://github.com/lede-project/source/pull/463#issuecomment-257545072 (and a ls -l /bin also confirms this, the "wget" tool is a symlink to uclient-fetch, not to busybox.)

I've just tested this on my guinea pig router, with PolarSSL library (not installed by default), but wget fails with "Invalid SSL certificate", because there is no certificate database in the default firmware (ca-certificates is the package name for that I think).

root@lede:/# wget -O /tmp/page https://wiki.openwrt.org/doc/howto/netfilter
Downloading 'https://wiki.openwrt.org/doc/howto/netfilter'
Connecting to 81.0.124.216:443
Connection error: Invalid SSL certificate

You can override the certificate check by giving it an option to not check certificates "--no-check-certificate", it is of course not a good thing to not check certificates, but if you need to download a package from dropbox or something like that in a pinch it will work fine from default LEDE firmware.

root@lede:/# wget --no-check-certificate -O /tmp/page https://wiki.openwrt.org/doc/howto/netfilter
Downloading 'https://wiki.openwrt.org/doc/howto/netfilter'
Connecting to 81.0.124.216:443
Writing to '/tmp/page'
/tmp/page            67344  --:--:-- ETA
Download completed (67344 bytes)
1 Like
38

@bobafetthotmail and @RangerZ You've identified a real problem. I throw down a challenge:

Can you come up with a straightforward, works-every-time procedure to help people who come to LEDE today?

  • What advice would you give your friend who wanted to try out LEDE?
  • Assume the friend isn't worried about time (give or take 15 minutes), what's the smallest number of steps that they would need to install successfully?
  • Does your procedure depend in whether they install additional packages?
  • Does the number of additional packages make a difference?

Thanks.

39

[quote="richb-hanover, post:38, topic:52, full:true"]Can you come up with a straightforward, works-every-time procedure to help people who come to LEDE today? [/quote]As-is LEDE isn't terribly friendly, so it isn't straightforward nor fast.

There are quite a few show-stoppers for a straightforward or fast "learn LEDE", one is that there isn't a decent way to see what functionality/packages are available (and their description) without installing LEDE first or looking at the source (or downloading/opening/reading the package lists manually), the other is that Image Builder works only on Linux 64 bit.

[quote]- What advice would you give your friend who wanted to try out LEDE? [/quote]Assuming a power user or better "skill level": set aside a few hours and install LEDE default images on a guinea pig router with USB port and 8 MiB or more internal flash, learn how to do extroot from the tutorial and then you can install whatever to try it out.

The pro way would be to try in a VM first so you can install packages and try out its functions from a safe environment. (It is possible to connect to the serial of a VM, plenty of tutorials, I've yet to get it to work on my linux system but that's another story).
Virtualbox can be set up to let people ssh/use the browser to control the VM too as if it was a router (there are tutorials), but again I've never done this myself.

The LEDE build system can generate pre-made Virtualbox disk images already, but currently the build bots aren't doing it, so there are no pre-made virtualbox images, you need to import the .img files in Virtualbox (easy to do anyway, check the "OpenWRT in Virtualbox" tutorial).

I also suspect that none made a dedicated LEDE configuration for virtualbox images as the one I made with the build system isn't getting network connection with default settings (and this is a bit bad admittedly).

  1. Determine what functionality/packages you need (VM? looking at a list?)

  2. use Image Builder to make a flashable image with these packages (since ImageBuilder works only on Linux 64bit most people will need to use a VM for this too, or maybe someone could try out the Windows 10 compatibility layer for linux applications)

  3. flash the image.

[quote]- Does your procedure depend in whether they install additional packages?[/quote]Yes. If they are satisfied with what comes with default image or little more (they need only luci webinterface), it makes more sense to just install the default firmware images and then install luci and whatever.

[quote]- Does the number of additional packages make a difference?[/quote]Unless someone goes very heavy-handed, it shouldn't be an issue for the Image Builder. Devices with 8 MiB of storage do limit you a bit, the ones with 16 MiB or more don't.

40

So to sum it up, we need either the Virtualbox LEDE to work decently first, or we need something that reads the package feeds and compiles a readable table/page/whatever with that.

Currently, the package lists used by LEDE's opkg tools are plain text files compressed with gzip in the usual linux tradition.

Each of them contains the various packages data, this is a random entry:

Package: px5g-standalone
Version: 2
Depends: libc
Source: feeds/base/package/utils/px5g-standalone
Section: utils
Maintainer: Jo-Philipp Wich <xm@subsignal.org>
Architecture: arm_xscale
Installed-Size: 25149
Filename: px5g-standalone_2_arm_xscale.ipk
Size: 25856
MD5Sum: 38a184399f37f25ecabb873cfd030454
SHA256sum: 3752198becdc4233094d829451ed5b8adf6706b5b9719798d1c574b10650c673
Description:  Px5g is a tiny standalone X.509 certificate generator.
 It suitable to create key files and certificates in DER
 and PEM format for use with stunnel, uhttpd and others.

I can make a bash shell script we can run on the wiki server to periodically download the same package lists opkg uses and then extract and load the relevant parts in something like the ToH to make them easily searchable at least.

That's the minimum imho.

EDIT: I started a new topic to discuss this as it is getting offtopic, here