Reproducible builds -- feeds

Dear all,

Does the feed architecture works like if I check out v17.01.4 and when I update and install feeds, do I get the same tree, or the latest?

I'm asking this hence I want to make reproducible builds.

Thanks,
Levente

See here

You'll need to "tag" the feeds and kernel trees if you want to be able to reproduce builds, at a minimum.

There is a belief that only the config.seed and the main tree reference is sufficient to recreate a build. However, as you suggest, knowing the state of all the component trees and any local changes is required to truly be able to recreate a build (assuming the build environment itself is unchanged). The script that creates an OpenWRT build reference is not reversible to the commit hash if you have any local modifications, and I have run across references on the master branch that are not reversible either.

I use a Python script I wrote that collects all the git tree commits, dirty status, and diffs, along with .config and a few other things and writes them to my images.

The advice linked by anomeone gives good reference. Well, I have written it, so it is good :wink:

But note also my last message in that thread:

For most users it is usually better to compile from the HEAD of the 17.01 branch instead of compiling the exact release like 17.01.1. When compiling from branch HEAD, the later fixes in the main 17.01 source and in the feeds get included into the build. To achieve that, do not checkout the release tag “v17.01.1” but checkout the branch “lede-17.01”:
git checkout lede-17.01
But there is no need to fetch an ancient release 17.01.4 to lock the feeds, if your only goal is reproducibility. You can also specifically checkout the current HEAD commit of the feed branches to lock the feeds.

  • checkout an exact commit of the main source (the current branch HEAD), but just store that commit hash for future.
  • update the feeds, and/or look up the HEAD commits from git servers
  • edit the feeds.conf.default to lock into specific feed commits, instead of just allowing feed to update. The same syntax with "^hash" as used in releases, works also otherwise.
  • update the feeds to fetch the exact commits that you specified

So editing this in feeds config

src-git packages https://git.lede-project.org/feed/packages.git

to

src-git packages https://git.lede-project.org/feed/packages.git^338690b2f79e2c7090be4e9adbb19b452c9e3c36

would lock the packages feed into the current 17.01 HEAD commit there.

Second alternative is to manually git checkout the specific feed commits instead of using the feeds update script.

Just store the references in your build and that makes it easier to later identify exact upstream source used. I use a rather similar approach as jeff, and my short file about commit info looks like this:

OpenWrt master r7050-9c409cb4e2 / 2018-05-27 19:54
---
main      2018-05-27 9c409cb kernel: add missing softdog symbol
luci      2018-05-27 2477544 Merge pull request #1833 from ysoyipek/pa
packages  2018-05-27 ff319fe php7: update to 7.2.6
routing   2018-05-17 23c0402 Merge pull request #378 from tohojo/bird-

Additionally I store all my changes using these scripts (that I also use to generate my R7800 and WNDR3700 community builds:

Ps.
17.01.5 is probably forthcoming in the next few weeks. As well as the first 18.06 rc builds.

Thanks all of your comments. It seems that feeds.conf.default at least at release commits does have explicit references. Like at v17.01.4 I have this:

src-git packages https://git.lede-project.org/feed/packages.git^cd5c448758f30868770b9ebf8b656c1a4211a240
src-git luci https://git.lede-project.org/project/luci.git^d3f0685d63c1291359dc5dd089c82fa1e150e0c6
src-git routing https://git.lede-project.org/feed/routing.git^d11075cd40a88602bf4ba2b275f72100ddcb4767
src-git telephony https://git.lede-project.org/feed/telephony.git^ac6415e61f147a6892fd2785337aec93ddc68fa9

For the time being, this is enough. I bet the same thing happens with the kernel.

I also rolled my own build script that checks out specific OpenWRT/LEDE version. I had a fear, that it is not enough, and I have to track feed changes.

Thank you very much pointing these stuff out.

Levente