I hear complains that maintaining Openwrt is a lot of work and people only make demands but does not contribute. I feel that it is going to be harder and harder to get more maintainers and contributions until the new comers are treated more fairly.
I would like to see the community flourishing instead of slowly dying.
Do you have a specific pointer on how people are treated unfairly? Note that demanding white space clean commits or Signed-off-by tags in commit messages are not "fuck around for no reason".
My experience was ok so far, the only problem i noticed are some absent package maintainers, which results into blocked PR.
Similar getting new packages or large PR merged can be hard, if you can't get/find a reviewer, thats when you need to promote your work and hop into irc/forum and grab one yourself.
They really need to work on being not hostile and clannish.
Can someone explain to me what was done wrong here? To me it seems like just standard HN crowd's kind of cargo cult, to blame everybody around, especially the FOSS projects.
This is unfortunately a problem for many open source projects. Reviewer resources are limited, and getting more reviewers is very hard. I believe it is a problem we just have to live with.
Contributing the perfect code you have worked on for months and not even getting an ack is of course frustrating. And even more so if you see other work merged in the mean time. It is easy to start thinking that this is "unfair". But it is important to keep in mind that no one is choosing to ignore you. We all need to think about how we can make the process as easy as possible for the reviewers. There are several ways to do that.
Collecting testers is one way. If you touch existing code, make sure you have gotten an existing user to test your changes.
Another way to make review easier is to split up the submission in managable, logically isolated, parts. You mention new packages or large PR as examples of submissions that can be hard to get merged, and there is a reason for that. This is simple resource allocation theory. It is very hard to get a big slot when competing with many small requests. It is necessary to have dedicated pools for different request sizes. Unfortunately, there is no "big review pool". So the best we can do is to split our submission into smaller parts. Maybe it is better so submit a new board without all the new drivers you wrote for initial review? Or maybe you can do your smaller code fixes and preparations first, and then add the new board?
New packages probably can't be split up significantly. And if they don't directly affect anything else, then there isn't much need for extensive regression testing either. But there are probably still things you can do to make review easy-peasy. Maybe add user documentation? Include your justification - why do you want this package in OpenWrt? Talk to existing users of the software you are packaging and see if anyone else is interested in the OpenWrt package.
Fix other stuff in the mean time. What can be considered unfair, but really is not, is that it helps to have a name the reviewers associate with high quality contributions. Fixing bugs in OpenWrt helps everyone, including yourself. Test PRs submitted by others to help both them and the reviewers.
Don't give up. And never assume anyone wants to be unfair.
For the record: I am a common OpenWrt user and not involved with the project at all.The above is just my personal thoughts based on observation of a number of open source projects.
I have witness hostility toward who is trying to help here. I think this is sad because the core contributors are exceptionally smart people and they could be of great help.
Now, when they want it, they do.
But the mantra that I keep hearing over and over is about "resource shortage". While I agree that it can be a problem, I also think this community does not seem to be doing well at attracting new long term contributors. And there is no strategy about it.
So this is a story:
It was the final of the Annual Lumberjack Competition, and after days of chopping, sawing, and tree climbing, only two competitors remained. One was an older experienced lumberjack and the other was a strong fellow about twenty years his junior. The rules of the competition were quite simple. The two lumberjacks would be sent into the woods and the one who could cut the most trees in eight hours would be the winner.
The younger lumberjack was full of enthusiasm and went off into the woods and started cutting trees immediately. He worked all through the day, barely taking time to catch his breath or to grab some food and water. He felt more and more confident with every tree he felled, because he knew that he had superior youth and stamina than the older lumberjack. He also felt sure of his win since he could hear the older fellow working in another part of the woods, and at regular intervals throughout the day, the noise of falling trees coming from the other man would stop. The younger lumberjack naturally assumed the older guy was taking more frequent breaks because of his age and lesser strength. He was confident his youth and stamina would win him the competition.
When the final whistle blew, the younger lumberjack felt confident he had won as he looked out over the piles of trees he had cut. He made his way over to the podium for the medal ceremony and climbed to the platform confident of his victory. The older fellow was there, actually looking much less tired than he did. When it was time to announce the winner, the younger lumberjack was waiting to hear his name, but was devastated when the older man was declared the winner.
The younger man turned to the winner and asked, “How can this be? I heard you taking breaks every hour while I worked continuously. I am younger, fitter, and stronger than you. How could you possibly have beat me?”
The older man smiled and said, “Son, I was not stopping to rest. I was stopping to sharpen my saw.”
Helping others to be able to help is a great way to sharpen the saw.
I have been a programmer for a long time. I love to code but I know that even if I worked 20 hours a day there is so much that I can do by myself. At a certain time, I understood that my time sometimes is better spent to attract talents and train them so they can be successful.
Main repo vs package are quite different in that regard...
It's probably hard to motivate people to keep contributing since feedback can take months, get completely ignored or that official ways of contributing are not for whatever reason used/accepted such as (https://github.com/openwrt/openwrt/pull/1950#issuecomment-548392685). You can find a lot more more examples by looking at the GitHub repo. As much as OpenWrt is on volunteer bases and what all that entails this will in most cases not leave a good impression and people will go elsewhere. Seeing that one of the key features with OpenWrt is slowly going away as more and more as consumer hardware gets more powerful and more storage is added you're also suddenly looking at "competition" from other similar distros.
Ynezz is trying to move everything to gitlab. Aside from that, the fact that most of the times pr gets ignored is a real problem... I think a real way to fix this would be create a group of moderators that have a more direct way to talk to the devs that have permission to push to master and tells them that a pr is ready once it has been reviewed.
There are lots of pr that has been abandon because either the reviewer or the submitter didn't respond.
This in the mailing list would have never happened because it gets immediate response.
OpenWrt has a HUGE number of devices it works for... many of them from an economic standpoint are worthless garbage relative to todays needs. On the other hand, there are no worthless garbage person-hours.
If OpenWrt picked say the top 15-20 most important devices, it seems it could probably support those devices with a lot more hours per device than it currently has to go around.
Even if it made a list of 15-20 "priority" devices, and continued to support the older devices on a "nice to have" basis.... it could improve things.
Saying no to new weird devices that only 18 people have is the right thing to do in my opinion.
As others have already mentioned, that's not a problem of OpenWrt alone. As an OpenWrt wiki admin, I have quite some issues with plugins which I report to the respective maintainer of that plugin. It sometimes takes months or even longer to get a first feedback.
If initial feedback is available, resolution of the issue might never happen.
Example 1:
Issue reported Aug 2018
Feedback in October 2018
Since then: Silence, issue is unresolved, no reaction to further messages
Example 2:
Issue reported Jun 2018
No feedback at all
I have other issues going back until 2015 which never got any feedback.
=> OpenWrt is not sticking out of the masses as extraordinary bad, since others are the same.
I don't understand why (most) FOSS people have mailing list fetish (fetish because there's no rational explanation for insistence of using it). Platforms like GitHub and GitLab provide a structured, organized and searchable way to submit contributions. It makes it easy to add labels, connect pull requests and issues, and it also generates fancy statistics (if needed).
Mailing lists, on the other hand, are just a bunch of text and its structure depends on the good will of the participants (adding proper tags in the subject), mail clients (which handle quoting) and are polluted by spammers. Browsing "threads" consist of clicking on previous/next links and hoping that you won't miss something once the discussion goes >>>>>>>>>>>>>>>>> levels deep and someone's mail client decides it's a good idea to insert some line breaks.
I wouldn't have anything against mailing lists if we didn't have something better, but relying on the 48 years old tech when we have things that makes everyone's life easier doesn't help in attracting contributors. That, and claiming that GitHub is "for scrubs". Antique wiki design perfectly matches with this.
Of course, OpenWrt isn't the only project with such issues, but it looks like the OpenWrt-LEDE split and remerge didn't help with anything.
That would be fantastic, but I wouldn't be surprised at all if that caused yet another fork or if it will be postponed indefinitely because of pure ignorance.
actually, the fish stinks right from the top of its head and in this case it is linux kernel itself. i opened a pr there once and short while after i got reply from bot to send it to mailing list. even some people later asked if i intend to upstream changes to linux kernel. no i don't-> pr exists, they should use technology as is, and not fuck around with contributors like some other guy said for a few core-members here
Many (early) OpenWrt developers are Kernel contributors, hence the preference for mailing list based workflow.
The resources required to host a Gitlab instance (or a commercial Github on premise one) are considerably higher compared to hosting a mailing list, that is one of the factors contributing to the decision (aside from the initial preference on MLs due to the familiar kernel.org workflow). Using free-to-use offerings is one possibility but not seen as a viable option by all team members. I personally am against using commercially 3rd party hosted infrastructure.
On the other hand it is correct that these platforms allow for a very good integration, as long as you adapt your workflow around them.
GitHub's barrier to entry is lower which both leads to an increased amount of contributions (good), a sometimes lower quality of contributions (not good) and raised expectations from the contributor side (depending on the pov, not necessarily good either).
There seems to be, among some people, the expectation that every PR made on Github is to be quickly merged and that any change requests to an open PR (such as adding S-o-b, alphabetical sorting of entries, consistent code style, white pace cleanup) is needless churn, specifically done to put off newbies. In my perception, some GitHub contributors think that editing kernel code suddenly is easy just because it is made via GitHub.
I have seen cases where PR creators got angry and cursing the reviewer because changes were demanded to a trivial patch. Maybe it is common in some parts of the GitHub culture to submit unclean patches and let the reviewer/merger take care of sorting out the details. Maybe it is the higher barrier of entry in mailing list based workflows that leads to only very dedicated people submitting issues and thus to a higher perceived quality of patches.
In any case, a lot of manual work is required yet to merge a (non-trivial) submission, combined with scarce developer resources (both in terms of number and time). Another negatively contributing factor is the unwillingness of some developers to master another workflow.
Things we did the last months to address these issues are:
onboard more developers
work on CI and PR automation as part of the GitLab evaluation
Digging out of this hole takes time however and is not something done in a matter of weeks for a larger project, especially considering OpenWrt's organization structure of a loosely coupled team of volunteers without strict governance and planning model.
There is the ability do do GitHub PRs now (slow merges does not mean its not possible. Merges are slow on the ML as well)
The amount of releases increased
The amount of supported targets increased
The amount of developers increased (not as much as we'd like though because other long term contributors becoming inactive)
The release process, albeit not yet as fast as we'd like, was improved compared to the state before
The forum was relaunched
Hosting sponsorship was acquired
Intangible assets have been transferred to a foundation
The number of commits in the time frame May '14 to May'16 [1] (before the fork) compared to May '16 to May'19 [2] (after the fork) increased by ~60% (from ~2100/year to ~3500/year)
The number of comitters to the the repository has been increased from 17 to 28 [3] [4]
The number of different contributors, albeit not clearly attributable to the pre-fork times due to the use if SVN, substantially increased as well from ~330 to ~660 [5] [6]
The project is using Git in the first place
A dedicated build infrastructure exists to provide binaries for all branches
The website is managed by contributors and not the team anymore
The reason email is preferred for patches is because if a git instance is set up correctly a correctly formatted patch once acked makes application of the patch automatic.
Github is in my humble opinion a mess. I find it a lot easier to work directly with a git repository.
The native technology to use with git for patches is email. A lot of people that use git seem to not to bother to read the documentation https://git-scm.com/doc
With all respect, instead of sticking the head in the sand and ignoring the numerous of rather negative experiences on behalf of both parts just drop GitHub if the majority thinks it's not worth bothering with and only accept patches using the mailing list. I think most expects that OpenWrt core works as a team at least somewhat rather than based on induvidual preference which makes the experience inconsistent and frustating. One of the reasons of the fork was to be more open and work with the community, that seems to have gone back more or less the "old" ways once again. That said, it's all volunteer work and so on but it generates unnecessary friction and it's unfortunately not the first time as GitHub users both good and bad have been more or less ignored and (indirectly) told that their work and effort isn't worth much. The mass closing of unhandled PRs during the transition back to OpenWrt wasn't the smoothest ride in that regard...
Looking at the Gitlab project, what's the official stance? Is it going to GitHub all over again but in a new dress or what's the plan?
It boils down to preference, while I understand that many don't want to change their workflow for both good and bad reasons and might be for a good cause in the end. The major downside with mailing lists is that it makes contribution obf code, review and tracking hard for many less seasonsed users. Keep mind that many younger users may not even have used mailing lists before which may turn them off right away. While many smaller projects uses GitHub or similar services there are also a lot of larger projects that uses Phabricator or similar for instance. That said, if you as a project want to attract more people you may want to listen to their opinions and take those into account.