Should OpenWrt/LEDE support devices with only 4MB Flash?

For Developers
56

4/32 warning in supported devices:
https://lede-project.org/supported_devices

4/32 warning in ToH:
https://lede-project.org/toh/start

4/32 warning in dataentries:
Where do you like the warning to be, top of the page or in the dataentry section?
https://lede-project.org/toh/hwdata/3com/3com_3crwer200-75
https://lede-project.org/toh/hwdata/3com/3com_3crwer100-75

The https://lede-project.org/meta/infobox/432_warning#details still need some refinement.

Please let me know your thoughts on all the above.

1 Like
57

[quote="RangerZ, post:55, topic:1018, full:true"]I tried it, and watched the videos. It does not appear to work the way you imply.[/quote]The profiles with the device names also have a list of packages installed for them, it's not just a copycat of profiles from the imagebuilder.
With a different config file in the server you can have
DEVICE1-openvpn
DEVICE1-luci
and so on (names are just examples), they guy in the mail said that it's also possible to make profiles for packages to be added (used by some downstream projects), but I've yet to test the whole thing.

selects the core packages and then walks the user through what file systems they may want. Do you want the Web gui? It's what we call in my business a "rules based configurator" which validates the component selections are properly made.

That was my goal too. Plan B (if Freifunk does not allow me to do that easily), is making a bunch of PHP pages with buttons and dropdown menus, they assemble the package list and send the command to imagebuilder in the server, then gives a download link to download the firmware.

I have no clue what the dependencies are.

As said elsewhere, dependencies are handled automatically as each package states its own dependencies. If this does not happen, it's a bug in the package or the package was not supposed to installed like that (like for the ath10k firmwares that currently are not set up in a convenient way for USB dongles).

I have been implementing Enterprise business solutions for almost 30 years now.

As already said somewhere else, with LEDE we have the luxury of having a more IT-literate target than average corporate drone that goes completely bananas if you move the icons on the desktop.

The less-IT-literate users will never ever even think about changing the firmware on an embedded device. They don't usually even know what is a firmware. So you can safely ignore them.

Indeed, I don't think it ever occurred to me when I built my fist OpenWrt device that the firmware might not fit. It wasn't until I started to look at travel routers that I learned there were issues.

There are a bunch of normal routers too that have issues now. In LEDE they changed other stuff in the system (for good reasons) and all devices with 32MiB of RAM can't use opkg to install packages at all as it goes OOM (out of memory) even if there are like 10 MiB of RAM free, and that issue affects far more devices than just those with 4MiB of flash.

Modern travel routers have 8 or even 16 MiB of flash anyway.

I suspect that most people want to download firmware images, install and go.

Which is why a web configurator making a firmware ready to be flashed is better here, no need to use clunky luci package install interface or SSH in and use opkg to install stuff. (if it is possible at all, and in many older devices it is not)

I think the idea of being able to do this online is great, but it's far from user friendly.

It's just a matter of devising a decent UI for it. If Freifunk's does not provide something decent I'm either hacking it (if I actually can) or making a bunch of PHP pages.

58

I overlooked that question in my previous reply.
To answer your question:

End of LEDE support can be expected around 2018
I wanted to be more precise than "in the future", to make clear that it might already be relevant for todays decisions whether to buy a device or rather not.
However, "around 2018" is sufficiently vague to cover a broad range of time.

1 Like
59

Looks nice, I think you should add that it's an upsteam issue not LEDE per se.

60

I think the above works well. It contains good language for novice or casual users and this same language, due to its wording, will be filtered by experience users. Works for me.

61

Top of the page! Eyes move from top to bottom. They will be able to see important messages first upon reaching the page

62

Please let me know your thoughts on all the above warnings.

Good, I like the warning in the device entry too.

I think the message is using complex words (we need to keep it easy for those that don't know english very well) and is partly wrong (the devices aren't unstable, they just can't add functionality), maybe something like

Devices with 4MB or less flash and/or with 32MB RAM or less can't install any packages with opkg or luci. They will only provide default LEDE features, which in most cases is just a bit better than stock firmware. See 432_warning for details.

The 432_warning is OK for me.

[quote="tmomas, post:58, topic:1018, full:true"]End of LEDE support can be expected around 2018
I wanted to be more precise than "in the future", to make clear that it might already be relevant for todays decisions whether to buy a device or rather not.
However, "around 2018" is sufficiently vague to cover a broad range of time.
[/quote]With all due respect to ldpinney and slh, that's just their own opinion. To make such official-looking statements we need to have a core dev that says it and I don't think they are core LEDE devs.

I personally don't think that 4MiB are in danger of getting dropped any time soon. Default functionality (usually already better than stock firmwares) will still fit for a long while, and if you remove Luci and use imagebuilder there is enough space for one payload application.

Same for most devices with 32MiB, they just become devices "for experts" or "for default functionality only", there is no reason to drop them for lulz.

63

not knowing who is who. Are these official statements from the project,
opinions from developers, or opinions from non-developers?

David Lang

64

I was thinking we should wait for one of the core people to chime in on that as well.

65

So here's the deal. LEDE is very limited in terms of developers hence there's not an option or value to deviate too much from upstream whether we like it or not unless we want to create some strange fork that's going to be a pain to maintain. The Linux kernel is growing, so are stock libs and applications. Work is being done to slim these down by default but again, no one is going to rewrite half of the distro's software and there's little to no interest upstream to maintain 8y+ (usually) old devices while adding new ones. There's just too much overhead, code-rot etc by doing so which more or less translates into that "everything" has to fit the new "way"/code. Also, having the above in mind we want upstream support when it comes to kernel/software etc, there's not enough manpower to reinvent the wheel and having support for a lot of custom software is a pain.

If you want to use your low-end router you have to accept that the newest and latest software wont run. 386 systems isn't support at all these days, I doubt 486 systems runs very well/at all on a recent kernel. You don't see ppl complaining about that...

66

So here's the deal. LEDE is very limited in terms of developers hence there's
not an option or value to deviate too much from upstream whether we like it or
not unless we want to create some strange fork that's going to be a pain to
maintain. The Linux kernel is growing, so are stock libs and applications.
Work is being done to slim these down by default but again, no one is going to
rewrite half of the distro's software and there's little to no interest
upstream so maintain 8y+ (usually) old devices while adding new ones. There's
just too much overhead, code-rot etc by doing so which more or less translates
into that "everything" has to fit the new "way"/code. Also, having the above
in mind we want upstream support when it comes to kernel/software etc, there's
not enough manpower to reinvent the wheel and having support for a lot of
custom software is a pain.

Nobody here is disagreeing with the above statements.

If the software grows too big to run then the device can't be supported any
longer.

What we are objecting to is the rush by some people to abandon/delete support
for devices.

There are always options to consider, smaller/lighter versions of software to
consider using, ways to NOT use some software.

As a couple of examples LEDE does not use glibc or systemd, in large part
because of the size of those packages. According to some people in the Linux
world, those would not be reasonable options to consider and any hardware that
wasn't big enough to run them isn't big enough to support at all.

But in some cases, we do need to look at tweaking/re-writing software. For
example, if opkg is unable to run on 32M ram, even if 10M ram is available, then
it's probably appropriate to find out why and figure out other ways to get the
job done. My uninformed guess is that opkg does something along the lines of
enumerating possible packages in ram, and as the list of supported packages
grows, it eats more and more ram. Options like mmapped files or using disk files
instead of ram are options if this is the case.

Nobody is objecting to dropping support for devices when they can't be loaded
with a minimum build, and nobody is objecting to dropping pre-built images for
systems when the standard image will no longer fit.

We are objecting to the idea that "someday it won't fit, so let's drop support
now".

We are trying to suggest a path for graceful degredation of support over time.

Personally, I see support as a continum, something along the lines of

  1. best support, able to run the standard image and load significant amount of
    additional packages

  2. limited support, able to run the standard image, but not able to load much in
    the say of additional packages.

  3. minimal support, no longer able to run the standard image, but able to run a
    "hello world" minimal image, so still useful for people doing customized things.

it seems as if the 4/32 routers are in category 2 right now. The imagebuilder
tool seems to be the key to making good use of these devices.

But for those saying that we should abandon support for everything that can't
run LUCI + Samba, etc. I'll point out that the project is "Linux Embedded
Development Environment", not "Linux Access Point OS". The project is intended
to support devices beyond just APs. As long as the hardware is able to boot a
minimalist build, someone can make use of it.

David Lang

1 Like
68

I'm doing to say best effort without doing intrusive changes to upstream source. We can always speculate about using but history shows that there's much talk and little work in that regard and it's not always desirable for newer systems and it can introduce limitations. I think tmomas is very realistic and fully agree with this warning.

1 Like
69

Unless you're willing to change the mind of devs upstream that's pretty much a fact, no one is going to stop you from using an older checkout.

1 Like
70

this should be changed to "4MB devices may not be able to install a GUI (LuCI)"

This information is geared towards new users not experienced one

71

As OP on this topic, I apologize that I have out of the loop for a couple days.

Here are a couple thoughts about what I've read in the discussion:

  1. Development for 4/32 devices will certainly continue for a long time. The build system is in place, the rough edges have been worn down, and they'll be stable for the forseeable future. The developer community are likely not affected. If those 4/32 devices are working for you, then there's no value in "throwing them off the cliff".

  2. But... The "support community" (those of you reading this note, for example) really care about getting people to purchase the right device.

  3. I think it's our responsibility to help new people with recommendations and forum, and website. I think it's safe to recommend that newcomers, or people who "Just want a LEDE router that works without a lot of farbling around" should get a router with 8/64 or better to start.

  4. Despite the reluctance to recommend specific hardware in the LEDE community, there's room to tell people that any new purchase should be 8/64 or better.

  5. I like @tmomas' work with the warnings. There also needs to be a clear explanation on the Quick Start Guide

  6. I also appreciate that we're abiding by Rule #12. Things have got a little heated, but steered way clear of unpleasantness. Thanks!

Update: Fixed to say 8/64 everywhere.

1 Like
72

First of all, I'm not pretending to speak for the LEDE team, however looking at the plain numbers presents a quite obvious situation.

Taking "Why no images generated for default D-Link DIR-600A1?" as an example (yes, some other 4 MB flash devices are a bit better than this particular specimen, the trend remains to be the same though).

D-Link DIR-600A1:
Backfire 10.03..............: 2293764 bytes Backfire 10.03.1............: 2949124 bytes Attitude Adjustment 12.09...: 2883588 bytes Barrier Breaker 14.07.......: 3276804 bytes Chaos Calmer 15.05..........: 3342340 bytes Chaos Calmer 15.05.1........: 3407876 bytes LEDE 17.01 release branch...: 3473412 bytes absolute firmware size......: 3735576 bytes maximum usable firmware size: 3538944 bytes
(all of these figures are for release images, including luci and a more or less identical feature set).

The erase block size for this (and most other) devices is 64 KB, so you now end up with 256 KB (== 4 erase blocks) free space, compared to 320 KB (== 5 erase blocks before). While this may look comfortable at a first glance, you have to consider that free space can only be assigned in (full) block size chunks, so once you touch the overlay partition at all, you already have one erase block in use (64 KB). Therefore the firmware creation tools used by LEDE enforce at least 3 erase blocks reserve for the overlay filesystem (that's where the maximum usable firmware size comes from, compared to the total size of the firmware partition). In other words, with 17.01 you'll only have 1 erase block (64 KB) before the hard limit, while 15.05.1 still gave you 2 spare erase blocks (128 KB) for your own use. On top of this there is the file system overhead needed for formatting to jffs2 as well (jffs2 does some light compression, but its fs header and log (more or less directory entries) need some space, then there is the hard requirement to keep some free space (in erase block == 64 KB chunks) for the garbage collection to work) at all times, reducing the usable free space even further. Around 25 KB are used by the configuration overlay immediately after firstboot, before you actually get a chance to configure anything.

Assuming pure statistics, what will the situation be in 12 or 24 months[1]?

No one has yet raised the suggestion to actually remove the hardware support for affected devices from LEDE's source repository. If you look deeper, you can still find full support for the Linksys WRT-54GL (4 MB flash, 16 MB RAM) in the repo, although support for devices with just 16 MB RAM had already been discontinued with Attitude Adjustment 12.09 and despite the fact that actually building a working up-to-date firmware for this device today is quite challenging[2].

This thread merely serves as a reminder for 'normal users', who expect to download and flash LEDE under the expectation to use it as a full featured replacement for the vendor firmware (including a webinterface, luci). Also keep in mind that luci is enabled for release builds, which means that it either fits into the firmware image (with the mandatory safety margin of at least 3 erase blocks) or no release images can be created for the affected devices[3], [4]. In my very personal opinion, this needs to be documented quite obviously, avoiding to raise false hope and expectations for (especially new) users.
Advanced users will obviously be able to delay the inevitable by quite some margin - depending on their use-cases and abilities to reduce LEDE's footprint below normal system requirements.

Despite the title, the RAM size is actually a much harder limit, affecting much more than just 4 MB flash devices. If you look through the commit log, you will notice quite some efforts to get opkg to use less RAM, but it's still a problem with just 32 MB RAM (even for normal operations, before touching opkg). Likewise you already are in trouble with sysupgrade and trying to flash a (larger) 6-8 MB firmware on a device with just 32 MB RAM, unless you really clean up services and loaded kernel modules manually beforehand, there is a high risk that you'll oom during the sysupgrade and brick your device for good (requiring bootloader/ tftp assistance to recover in the best case).

[1] switching from mach files to device tree (post 17.01) offers a potential to free up a little space in the kernel, but this isn't a whole lot and will probably be eaten more or less completely by upgrading to kernel 4.9+ (also keep in mind that the FDT file needs to be appended to the kernel image, which might or might not compress as well as the mach files before), so don't expect a significant positive effect from this switch.

[2] no webinterface, better no pppd, quite some custom configurations to get the kernel's runtime memory requirements as small as possible, disabling whatever is humanly possible (definately no IPv6, better no wireless either).

[3] with LEDE's pretty new feature of device specific rootfs images, it would technically be possible to decide between installing luci or omitting it on a per device basis, e.g. based on flash sizes, but support for something like this hasn't been implemented yet and would require quite some attention (both as in source patches and tagging of device classes) from interested parties. As implemented right now, the decision is binary - either the default (release) config (including luci) fits XOR building a firmware image for the affected device fails and no firmware will be available.

[4] I would expect that there already are a couple of 4 MB flash devices in the target list for which no release firmware can be built for 17.01, because of less ideal flash partitioning schemes chosen by the vendor (dropping free space below 3 erase blocks). Those are probably a minority, but given the close numbers, I'd be very suprised if there wouldn't be any affected.

3 Likes
73

Thanks slh that is a very thorough assessment.

74

[quote="metai, post:67, topic:1018, full:true"]Wait, are you serious? At the moment there is no official release build for any device (as of this post, the buildbot has been building packages for 17.01 for the last few days), and there is no word from any dev in the matter. And yet you are making the blanket statement "4MB devices won't be able to install GUI (LuCI)" on the official web page.[/quote]I actually wrote that sentence as a stopgap until we could make something better. It's technically true. Devices with 4MiB can't install a damn thing through opkg, especially 250-ish KiB packages like luci, and it's not exactly news. Luci is the most sought after package most newbies want to install after flashing LEDE so it's good enough for a stopgap.

As you may notice, it has been replaced with another statement by tmomas and I currently proposed a more elaborate error message in this post that will clear that "my 4 MB device will never have a GUI" issue you raised

But in general yes, we need a clear warning for newbies, there are many devices that are still technically supported but are not recommended, and we need to make sure newbies stay clear of them (because seriously, with like 20$ you can buy stuff with 128MiB RAM and 16 MiB flash).

75

@slh Good assessment with facts, that's much better than a random opinion.

@tmomas how about cloning slh's analysis in the article where we explain the issue in the warning messages ? A paragraph like "In-depth analysis of the issue" or something.

(note that I'm not asking you to do it, I ask your opinion here. I can move things to wiki too)

76

Also big yes to this. @tmomas do we have a field for this in the ToH? I remember that one of the goals in LEDE ToDo was to categorize the devices in support classes like this.

1 Like