Quality of Openwrt builds - Testing required

How much testing is done before commiting code ? usign broken because of a LD_LIBRARY_PATH or a simple lack of including the required limits.h on the pcap_usb.c ! this is simple stuff that if a developer tests he/she should find defects and this is not about running the code is about building it!

It is continuously been built for all devices

http://release-builds.openwrt.org/18.06/images/waterfall

@mbo2o

18.06 builds are totally different than the bleeding edge master. There are separate buildbots for master, 18.06 and old 17.01 branches. So, no relevance to the LD_LIBRARY_PATH ucert problem in master, that was mentioned by OP.

@rmandrad
Master, the bleeding edge, can be broken sometimes if some commits broke things. Master is meant for development & testing by users/developers who are willing to accept the occasional brokenness.

Right now master has problem with new ucert, as discussed in

But even that can be bypassed with the patch proposal in that PR. Builds can be done again, once you have created the ucert keys once, e.g by applying that patch from the PR.

Likely a fix will come soon, or the ucert based signing will be reverted for a while.

I have the workarounds for both ucert and pcap I accept the bleeding edge master uncertainty... however how these buildbots pass their builds? these errors are platform independent, they are very basic !

They don't always pass
http://phase1.builds.lede-project.org/waterfall

regarding the ucert problem, possibly the buildbot managed to build certificates first, before some later change broke things

https://git.openwrt.org/?p=openwrt%2Fopenwrt.git&a=search&h=HEAD&st=commit&s=ucert

I have noticed also with my own builds that once I had created keys (with patch from that PR), the build again worked even if that patch was not present any more.

You mean that the buildbots build incrementatlly ? if so - shouldn't they do full builds ? unless one is to use a base build followed by incrementals

I feel that there is an opportunity here to improve the process

Nowadays.
That was one of the big changes 2 years with the LEDE architecture, as the old Openwrt strategy of "build everything from scratch everything for every target" led to buildbot crunching each target only 1-2 time per week. Now the firmware images are built 1-2 times per day if there are changes, and the same for packages in phase2 buildbot that builds packages with the SDK built in phase1

Both of those cleanup the build dir after a few days and rebuild everything. But not daily.

And the build signing key is meant to be rather permanent in any case. At least I do all my builds using the same keys in all of my builds.

I think that the signing keys for buildbots have been create earlier (and they are download during the build run). See steps 32-33 here: http://phase1.builds.lede-project.org/builders/ar71xx%2Fgeneric/builds/880

thank you for the explanation - much clearer about the process