Communication problem with the OpenWrt team

As others have already mentioned, that's not a problem of OpenWrt alone. As an OpenWrt wiki admin, I have quite some issues with plugins which I report to the respective maintainer of that plugin. It sometimes takes months or even longer to get a first feedback.

If initial feedback is available, resolution of the issue might never happen.

Example 1:

  • Issue reported Aug 2018
  • Feedback in October 2018
  • Since then: Silence, issue is unresolved, no reaction to further messages

Example 2:

  • Issue reported Jun 2018
  • No feedback at all

I have other issues going back until 2015 which never got any feedback.

=> OpenWrt is not sticking out of the masses as extraordinary bad, since others are the same.

I don't understand why (most) FOSS people have mailing list fetish (fetish because there's no rational explanation for insistence of using it). Platforms like GitHub and GitLab provide a structured, organized and searchable way to submit contributions. It makes it easy to add labels, connect pull requests and issues, and it also generates fancy statistics (if needed).

Mailing lists, on the other hand, are just a bunch of text and its structure depends on the good will of the participants (adding proper tags in the subject), mail clients (which handle quoting) and are polluted by spammers. Browsing "threads" consist of clicking on previous/next links and hoping that you won't miss something once the discussion goes >>>>>>>>>>>>>>>>> levels deep and someone's mail client decides it's a good idea to insert some line breaks.

I wouldn't have anything against mailing lists if we didn't have something better, but relying on the 48 years old tech when we have things that makes everyone's life easier doesn't help in attracting contributors. That, and claiming that GitHub is "for scrubs". Antique wiki design perfectly matches with this.

Of course, OpenWrt isn't the only project with such issues, but it looks like the OpenWrt-LEDE split and remerge didn't help with anything.

That would be fantastic, but I wouldn't be surprised at all if that caused yet another fork or if it will be postponed indefinitely because of pure ignorance.

3 Likes

actually, the fish stinks right from the top of its head and in this case it is linux kernel itself. i opened a pr there once and short while after i got reply from bot to send it to mailing list. even some people later asked if i intend to upstream changes to linux kernel. no i don't-> pr exists, they should use technology as is, and not fuck around with contributors like some other guy said for a few core-members here

1 Like

Many (early) OpenWrt developers are Kernel contributors, hence the preference for mailing list based workflow.

The resources required to host a Gitlab instance (or a commercial Github on premise one) are considerably higher compared to hosting a mailing list, that is one of the factors contributing to the decision (aside from the initial preference on MLs due to the familiar kernel.org workflow). Using free-to-use offerings is one possibility but not seen as a viable option by all team members. I personally am against using commercially 3rd party hosted infrastructure.

On the other hand it is correct that these platforms allow for a very good integration, as long as you adapt your workflow around them.

GitHub's barrier to entry is lower which both leads to an increased amount of contributions (good), a sometimes lower quality of contributions (not good) and raised expectations from the contributor side (depending on the pov, not necessarily good either).

There seems to be, among some people, the expectation that every PR made on Github is to be quickly merged and that any change requests to an open PR (such as adding S-o-b, alphabetical sorting of entries, consistent code style, white pace cleanup) is needless churn, specifically done to put off newbies. In my perception, some GitHub contributors think that editing kernel code suddenly is easy just because it is made via GitHub.

I have seen cases where PR creators got angry and cursing the reviewer because changes were demanded to a trivial patch. Maybe it is common in some parts of the GitHub culture to submit unclean patches and let the reviewer/merger take care of sorting out the details. Maybe it is the higher barrier of entry in mailing list based workflows that leads to only very dedicated people submitting issues and thus to a higher perceived quality of patches.

In any case, a lot of manual work is required yet to merge a (non-trivial) submission, combined with scarce developer resources (both in terms of number and time). Another negatively contributing factor is the unwillingness of some developers to master another workflow.

Things we did the last months to address these issues are:

  • onboard more developers
  • work on CI and PR automation as part of the GitLab evaluation

Digging out of this hole takes time however and is not something done in a matter of weeks for a larger project, especially considering OpenWrt's organization structure of a loosely coupled team of volunteers without strict governance and planning model.

  • There is the ability do do GitHub PRs now (slow merges does not mean its not possible. Merges are slow on the ML as well)
  • The amount of releases increased
  • The amount of supported targets increased
  • The amount of developers increased (not as much as we'd like though because other long term contributors becoming inactive)
  • The release process, albeit not yet as fast as we'd like, was improved compared to the state before
  • The forum was relaunched
  • Hosting sponsorship was acquired
  • Intangible assets have been transferred to a foundation
  • The number of commits in the time frame May '14 to May'16 [1] (before the fork) compared to May '16 to May'19 [2] (after the fork) increased by ~60% (from ~2100/year to ~3500/year)
  • The number of comitters to the the repository has been increased from 17 to 28 [3] [4]
  • The number of different contributors, albeit not clearly attributable to the pre-fork times due to the use if SVN, substantially increased as well from ~330 to ~660 [5] [6]
  • The project is using Git in the first place
  • A dedicated build infrastructure exists to provide binaries for all branches
  • The website is managed by contributors and not the team anymore

There is ongoing work to refresh the project CI.


[1]: expr $(git log --all --since=2014-05-01 --until=01-05-2016 --oneline | wc -l) / 3
[2]: expr $(git log --all --since=2016-05-01 --until=01-05-2019 --oneline | wc -l) / 3
[3]: git log --all --since=2014-05-01 --until=01-05-2016 --format="%cN" | sort -u | wc -l
[4]: git log --all --since=2016-05-01 --until=01-05-2019 --format="%cN" | sort -u | wc -l
[5]: git log --all --since=2014-05-01 --until=01-05-2016 | grep Signed-off-by | sort -u | wc -l
[6]: git log --all --since=2016-05-01 --until=01-05-2019 | grep Signed-off-by | sort -u | wc -l

7 Likes

I just needed a manufacturer in the ToH... too much too ask? 6 months, nothing... scrub material?

You could have sent a reminder. There's always the chance that a posting just gets overlooked.

Apart from that, you can always create pages on your own, without using the page-creation mechanism

3 Likes

The reason email is preferred for patches is because if a git instance is set up correctly a correctly formatted patch once acked makes application of the patch automatic.
Github is in my humble opinion a mess. I find it a lot easier to work directly with a git repository.

The native technology to use with git for patches is email. A lot of people that use git seem to not to bother to read the documentation https://git-scm.com/doc

With all respect, instead of sticking the head in the sand and ignoring the numerous of rather negative experiences on behalf of both parts just drop GitHub if the majority thinks it's not worth bothering with and only accept patches using the mailing list. I think most expects that OpenWrt core works as a team at least somewhat rather than based on induvidual preference which makes the experience inconsistent and frustating. One of the reasons of the fork was to be more open and work with the community, that seems to have gone back more or less the "old" ways once again. That said, it's all volunteer work and so on but it generates unnecessary friction and it's unfortunately not the first time as GitHub users both good and bad have been more or less ignored and (indirectly) told that their work and effort isn't worth much. The mass closing of unhandled PRs during the transition back to OpenWrt wasn't the smoothest ride in that regard...

Looking at the Gitlab project, what's the official stance? Is it going to GitHub all over again but in a new dress or what's the plan?

1 Like

It boils down to preference, while I understand that many don't want to change their workflow for both good and bad reasons and might be for a good cause in the end. The major downside with mailing lists is that it makes contribution obf code, review and tracking hard for many less seasonsed users. Keep mind that many younger users may not even have used mailing lists before which may turn them off right away. While many smaller projects uses GitHub or similar services there are also a lot of larger projects that uses Phabricator or similar for instance. That said, if you as a project want to attract more people you may want to listen to their opinions and take those into account.

1 Like

if the majority thinks it's not worth bothering with and only accept patches using the mailing list.

Lets see what majority thinks.

Looking at the Gitlab project, what's the official stance?

Pending, see 4. in combined voting.

only devs can partecipate to the vote or everybody ?

According to the rules (https://openwrt.org/rules) commit access is the requirement for voting.

1 Like

Perhaps it would make more sense to ask the community first since that's what OpenWrt struggles with and based on the results look at what's viable?

4 Likes

Anyway I really think platform like github or gitlab should NOT be dropped... They make patch review so much easier and organized than mailing list...

2 Likes

Perhaps it would make more sense to ask the community first since that's what OpenWrt struggles

I've decided to kill two birds with one stone instead (cheated, took a shortcut, name it as you wish). I've instead decided to ask developer community, team members, fellow developers, people already using the current tools and let them make the binding decision by actually voting for their favorite. So we get survey and the usable results, which would translate into actual action in one go.

I did so, because I think, that if this established voting mechanism is good enough for other topics, it should be good enough for this decision as well.

On the other hand, I've decided to ask wider community about the new project logo/colors, because they're actual users of this change, they're going to look at it for the next few years :slight_smile: The result and feedback was amazing, so I plan to use this kind of decision help in such suitable topics in the future as well.

BTW evaluation of GitLab as next platform has started around 7/2019, right after the Hamburg 2019 developer meeting, IIRC you were aware about that intention since it was discussed in public since then and you've even suggested to do such survey, but actually done nothing in this regard by yourself. Don't blame me now.

Anyway I really think platform like github or gitlab should NOT be dropped

There is no such option in the voting email, or is there? The question is: What should be our next development platform? I don't see anything about closing GitHub. GitLab.com/openwrt is for evaluation only since the beginning and could be disabled, that's correct. FlySpray is probably going to be exchanged with the issue/ticket feature in the selected platform. Patchwork OR similar mail based alternative is going to remain, this is 99.999999% sure :slight_smile:

It's really about having somehow blessed path/direction, because other planned changes (issue and review bots for example) would mean tighter integration with the selected platform and nobody wants to invest time in the almost "phased out" platform, thus throwing some non-trivial work away.

2 Likes

I've been invited to core dev meetings but I've declined as by the time I had lost interest due multiple reasons. As for GitLab I personally don't see what it solves except for adding more hurdles (although not major ones) for contributing compared to GitHub however I do know there's a rather strong desire for having as much as possible self-hosted so I guess it ticks that box at least. In addition at the time I looked at GitLib it didn't seem to have any support for handling patches via e-mail so there would be one "patch repository" which made it look even less of a reasonable move except just for the sake of it. I'm not totally opposed by the idea but I don't see the benefits making it worthwhile including the administrative time that needs to be allocated.

That said, I'm considering myself as average at best when it comes to git and around amateur level when it comes to reviewing and coding so that may very well impact on my impression and ability to interract with tools.

I personally believe that one of the requirements should be ability to have a unified patch repo/management as it's clearly causing many issues and should be prioritized.

It's a mixed bag. Some things go well, others do not.

Regarding the packages feed, I've been trying to get as many PRs cleaned up and merged. At least get them under 50. It's quite difficult.

Same with the issues there.

1 Like

Are the results on that poll in and if so publicly available? The mailing list implied a voting count last weekend but there appeared two votes later on the mailing-list that do not seemed mapped in https://cryptpad.fr/code/#/2/code/view/PvVBSUc8jbD8gQxcOmfrwudZCA+0eD7Xbp53FrG+5uk/embed/present/

Are the results on that poll in and if so publicly available?

http://lists.infradead.org/pipermail/openwrt-adm/2020-February/001316.html

4 Likes