Criteria for first LEDE stable release?

For Developers
3

Based on the past 3 months I would pick 3 milestones.

  1. General tree cleanup and merging of the backed up openwrt patch queues - this seems to have slowed weeks ago, basically done?
  2. Ath9k buffer bloat improvements and added coolness - I've been running the airtime fairness patch for weeks, I don't know if this ever left staging. Would be nice if it got in.
  3. Release tooling - @jow seems to have slowed down so I'm guessing he's nearly happy with things

As soon as the tooling is done I'm guessing we'll get a beta release. Can't see anything else we need to wait on. Would be cool to have a Christmas day or New years day final release.

We decided against boose codenames right?
I think "Reboot" is fine for LEDE1.0, but has any discussion happened over what the next codename meme is going to be?
Dank memes are important for long running projects :slight_smile:

4

FYI, for those who don't know, my perspective is from one whose normal job is a product manager, not a developer. My usual role is sitting with the developers and, based on the feedback from "the market," saying "this is great for now, please finish that for this release, and we can leave the other thing for a future release."

I think the current feature set, based on what I have seen (I do not claim full knowledge of the features of LEDE), are quite sufficient for a release. I think the goal of a "release" for a "product" like LEDE should be to provide a well-developed, well-tested, well-documented "snapshot" of the development at a particular point in time for others who want to either just put it on a device to use or to use as a stable starting point for additional development. I have to agree with weedy, though, that getting the bufferbloat improvements into the release would be a very good thing.

Into the future, I think it would be useful to simply choose a point in time for a "proposed" release, and evaluate the ongoing projects at that time. If the core features are in good shape, then we push to a release based on rough agreement on what's in, what's out, and what needs to be hurried up into the release. The time (every 6 months? 3 months?) just stands as a flag on the horizon so everyone has a timeframe to work to (if I want this in the next release, I need to get it integrated by May for the June release...).

Having releases far apart means the releases are more consequential - more new features and capabilities - but making them closer together decreases the penalties for "missing" the release, which I believe increases the overall quality of the code in the release. More frequent releases are a boatload of work, but, if you do it frequently, it becomes much more routine. The further apart the releases are, the more "startup overhead" there is in the release mechanism.

The other thing that frequent releases potentially help with is back-porting. If there's a critical bug found, back-porting, as ugly as it is, is probably necessary. However, with frequent releases, you can just back-port to the most recent release and go forward, because nobody needs to be "married" to a year-old release to maintain compatibility.

So my two cents worth is this: let's just do a release ASAP (preferably with the bufferbloat stuff in, if everyone agrees it's stable and working properly) and then do small releases every 3 months -in other words, have a fairly constant release cycle, with one going out the door and one coming up on the horizon all the time. Let's automate it (to the maximum extent possible) and minimize back-porting.

-Bill

5

+1

The sum of all the small improvements that have been done since the reboot is more than sufficient reason to release. I think it would be good to have a new "baseline" for further development now. If the infrastructure is ready, then I say: release asap.

If my opinion as more of a user than developer counts for anything :wink:

6

I believe the understanding is that the first release will happen when there's an infrastructure in place to support release. Can't happen sooner than that. :wink:

7

What's missing?

8

Few days ago i posted this question:
https://forum.openwrt.org/t/who-wants-to-use-lede-and-is-still-not-using-it-and-why/509/2

Maybe the replies that I get there could shine a light on what certain end users like to see for LEDE's first release.

9

+1

The sum of all the small improvements that have been done since the reboot is
more than sufficient reason to release. I think it would be good to have a
new "baseline" for further development now. If the infrastructure is ready,
then I say: release asap.

IMO, there would have been value in making a LEDE release as soon as the
branding change from OpenWRT to LEDE was completed, even with no other
development work, just to give people a place to start with before doing any
other development.

beyond that, a release a few months later to include the initial burst of
cleanup work would have also been very nice to see.

There is no need to wait for something "significant" enough to be a release,
LEDE has that many times over.

David Lang

1 Like
10

Hear hear. In fact, I can imagine that collecting all improvements over the last OpenWrt release is itself quite a task. Just look at how many new devices are supported, how many libraries have been updated, the number of structural and organizational improvements. The "list of improvements" that would go into the release notes would be massive already.

11

I flashed a firmware today and it changed from CURRENT to SNAPSHOT :smile: how fast :laughing:

12

For me, I'd love to see the bufferbloat improvements on Ath9k in the first release. But that's just me. Feature-wise I think LEDE is fine at the moment for its first release. The most important thing is a well-tested and thus stable first release that can be used in home networks without fear of using a unstable snapshot on accident.

13

For the record, I would like current versions of both bufferbloat and make-wifi-fast/airtime-fairness fixes to be incorporated.

They would bring demonstrable benefits to the initial LEDE release since they dramatically decrease latency/lag and increase wireless performance with essentially no downsides.

My impression (which I'm sure will be vetted by those teams) is that:

  • Bufferbloat/fq_codel/cake/etc. are in very good shape and are included by default.
  • Make-wifi-fast/Airtime-fairness are also stable and built into multiple independent/community builds. Even though there may be further improvements to be had in the next few months, we would be well-served to have them in place.
14

@Mushoz

So this would bring in a Release Candidate test cycle?

Is there a test checklist or something that we can use to make sure important stuff is actually being tested?
What do the DEVs want to know? What can end users do? Would it be possible to have some tests automated? Not sure if that makes any sense, the idea just popped into my mind just now.

15

@charcoal I am not a developer, so I'm not the right person to ask. Just an enthusiastic user giving his opinion. I'm more than willing to help test release candidates once it comes to that.

16

And if so. Will this result in a feature freeze to "force" only commits that improve what is already there to be merged/cherrry-picked into the stable branch?

17

[quote="steinmb, post:16, topic:552"]

So this would bring in a Release Candidate test cycle?

Will this result in a feature freeze to "force" only commits that improve what is already there to be merged/cherrry-picked into the stable branch?
[/quote]Pretty much so on the release branch. but development goes on in the master. The release branch only gets fixes and possibly a few important changes/features.

Based on experience with Openwrt releases (AA, BB, CC), the rc phase tends to get a bit too prolonged and both devs' and enthusiasts' interests wander toward the new functionalities that are implemented in the master at the same time.

The release/branching automation work done recently by @jow should enable a more rapid rc/release cycle than earlier. And that is a major benefit as a long rc phase tends to move into a non-event.

18

Another factor to consider is that the Release Candidates should be widely-publicized as stable, near final, with testing encouraged.

I wasn't particularly focussed on the CC release process, but my impression was there were a few mentions from official sources of the RC1, RC2, and without a bold proclamation of what's supposed to be working.

That made it hard for me to decide "when to jump in" and try out the release candidates. Our release notes need to ask people to test, and give them reasons to do so.

I would advocate that our Release Notes for RC1 &c have the form:

  • The LEDE Project provides a powerful and stable operating system replacement for routers and embedded devices. For more information, see https://lede-project.org/about
  • This is Release Candidate #1 (RC1) for our upcoming "LEDE XX.YY" release. It contains the following new features:
    • Feature A
    • Feature B
    • ... etc...
  • We believe the software is stable, and can be used widely, although still in test situations.
  • If no bugs were found, we would ship this version. (Of course, there probably will be bugs in this release candidate, so...)
  • We plan to make an additional N Release Candidate releases, and ship on or around ddMMMyyyy
  • No new devices (beyond those already in the ToH) will be added/supported in this release
  • Please help us test!
  • [And finally, no talk about the new development/trunk release. We don't want potential testers distracted by shiny new toys that won't appear for months.]
19
  • This is Release Candidate #1 (RC1) for our upcoming "LEDE XX.YY" release. It contains the following new features:
  • Feature A
  • Feature B
  • ... etc...

we need volunteers to dig through the git changelog to identify what features
have been added that are worthy of being listed. One problem with releases of
distros that combine a lot of upstream changes is that the most important thing
is either support for new devices (i.e. someone can't run the earlier version)
or just the fact that it includes newer version of a lot of software, and all
the new stuff that implies.

  • If no bugs were found, we would ship this version. (Of course, there probably will be bugs in this release candidate, so...)
  • We plan to make an additional N Release Candidate releases, and ship on or around ddMMMyyyy

I would replace this with something like:

If there are no bugs found in this rc, we plan to ship it as the release on or
around ddMMMyyyy.

If there are bugs found, we will ship an additional rc instead.

David Lang

20

Agree with everything that @dlang said. In fact the list should have up to a dozen big features, with a final bullet item pointing people to a longer, fine-grained list of all the changes/bug fixes/etc.

With this initial LEDE release, we might forgo the list of "new devices supported", since that's the universe of devices. But that probably should become a component of subsequent releases.

21

Hmm... Just as a remainder, link to the CC15.05 RC1-RC3 release notifications:
https://lists.openwrt.org/pipermail/openwrt-devel/2015-May/033124.html
https://lists.openwrt.org/pipermail/openwrt-devel/2015-June/033623.html
https://lists.openwrt.org/pipermail/openwrt-devel/2015-July/034444.html

Pretty detailed info, plus change notes between RC1 - RC2 - RC3

And the same info was also published on forum. RC1 and RC2 messages have been overwritten by the RC3 message, but both it and the final release message are still there:
https://forum.openwrt.org/viewtopic.php?id=57453
https://forum.openwrt.org/viewtopic.php?id=59528

I think that sets the "expected" standard for the LEDE release notifications.

22

Thanks, @hnyman for refreshing our memories about the CC release notes.

You're right about the detailed change/update info. Those were well described. But the release notes above, from May, June, and July 2016 don't give any indication of a time frame for when the software would be "done", nor about the expected stability of each release.

Without those cues, it was hard for me to judge where the builds crossed my willingness for experimentation/threshold for pain to know when to jump in.

I am hopeful that the LEDE release notes will set the expectation that the firmware is stable, we would release as-is (modulo bugs) and real soon, and contain a corresponding exhortation to start testing now.