Checkout a tagged release's source code (like v18.06.1)

I am sure that sooner or later there will be a flood of questions how to checkout the exact release source code for a tagged release (like 17.01.0-rc1), so this is a short guide:

You need to have already cloned the LEDE source repo in advance with
git clone https://git.lede-project.org/source.git

Then you only fetch the tags and checkout the correct one:

git fetch --tags
git checkout v17.01.0-rc1

Example workflow how to fetch v17.01.0-rc1 source:

perus@ub1610:/r7800-lede1701$ git fetch --tags
remote: Counting objects: 19, done.
remote: Compressing objects: 100% (17/17), done.
remote: Total 19 (delta 13), reused 0 (delta 0)
Unpacking objects: 100% (19/19), done.
From https://git.lede-project.org/source
   31b0640906..e038c60049  lede-17.01   -> origin/lede-17.01
 * [new tag]               v17.01.0-rc1 -> v17.01.0-rc1

perus@ub1610:/r7800-lede1701$ git tag -l
reboot
v17.01.0-rc1

perus@ub1610:/r7800-lede1701$ git checkout v17.01.0-rc1
A	files/etc/Compile_info.txt
A	files/etc/applyHNsettings.sh
...
M	target/linux/ipq806x/base-files/etc/board.d/01_leds
Note: checking out 'v17.01.0-rc1'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at 68a04dbb42... LEDE v17.01.0-rc1: adjust config defaults

Below is verification that the main source code is at the release tag and that all feeds point to a specific commit instead of the moving HEAD of the respective lede-17.01 branch in that feed:

perus@ub1610:/r7800-lede1701$ git log --oneline | head -n 5
68a04dbb42 LEDE v17.01.0-rc1: adjust config defaults
ec095b5bf3 ledtrig-netdev: don't cancel work on events for different interfaces
b8fcbbf31d kmod-sched-core: Add HTB and TBF traffic shapers
70a6bbd53d ar71xx: prepare jffs2 partition properly in factory.bin for BHR-4GRV2
f5e8c908bd mxs: remove stale references to obsolete kernel module packages

perus@ub1610:/r7800-lede1701$ cat feeds.conf.default 
src-git packages https://git.lede-project.org/feed/packages.git^31d89be9e69bac261bfe7440512cb4e0f3356255
src-git luci https://git.lede-project.org/project/luci.git^472dc4b9e2ca71c114f5da70cb612c1089b8daa7
src-git routing https://git.lede-project.org/feed/routing.git^a6c7413594a0e4b42dab42bb5fa68534e39b7d0c
src-git telephony https://git.lede-project.org/feed/telephony.git^1f0fb2538ba6fc306198fe2a9a4b976d63adb304
8 Likes

EDIT:
This advice mainly concern only 17.01.0-rc1, as the tagging script has been modified for rc2 to create and include version.date and version files in the archive. Those will set the correct date and git version. (but will also prevent recognising possible later git commits, as they override the git versioning.)

Note that if you did not use git, but download the release sources as an archive file from Github, some of the version info does not get set and some packages may fail to compile to due to missing date/time reference:

Easy fix:
Run this command in the build root directory to create the timestamp file "version.date" there:

date +%s > version.date

Developers will likely fix this for the next releases, so the fix is likely needed only for rc1.


Explanation:

The Makefile logic (in include/toplevel.mk and scripts/get_source_date_epoch.sh) expects to get a value value from git for the revision and hash as well as SOURCE_DATE_EPOCH, but if you are not in a git repo, those will be left unset. And that then triggers the error in those packages that use the SOURCE_DATE_EPOCH variable without verifying that it is set properly.

I tested myself and both getver.sh and get_source_date_epoch.sh fail to provide a version/time reference when sources from archive are used:

$ wget https://github.com/lede-project/source/archive/v17.01.0-rc1.tar.gz
$ tar -xvf v17.01.0-rc1.tar.gz
$ cd source-17.01.0-rc1/
$ ./scripts/getver.sh 
unknown
$ ./scripts/get_source_date_epoch.sh 
      <-- empty result

@jow
I wonder if the tag commit should also include a "version.date" file for scripts/get_source_date_epoch.sh and possibly a "version" file for scripts/getver.sh as a fallback. The build system rather bluntly assumes that sources have been got via git, but that may not be true. I think that build000 already ran into this issue as he wrote two packages bugs about the failures.

2 Likes

@hnyman - will look into providing fallbacks for the epoch date asap

have compile issue:

wget https://github.com/lede-project/source/archive/v17.01.0-rc1.tar.gz

tar -xvf v17.01.0-rc1.tar.gz

cd source-17.01.0-rc1/

scripts/feed update -a
scripts/feed install -a

make defconfig
make prereq
make menuconfig

Target System (Lantiq)
Subtarget (XRX200)
Target Profile (o2 Box 6431 / Arcadyan VGV7510KW22 (NOR))

Base system -> block-mount
sqm-scripts

Firmware -> dsl-vrx200-firmware-xdsl-a-patch
dsl-vrx200-firmware-xdsl-b

Kernel modules -> Filesystems -> kmod-fs-ext4
USB Support -> kmod-usb-storage

Languages -> Perl -> Perl
-> perl-html-tree
-> perl-http-message

LuCI -> Collections -> luci
-> Applications -> luci-app-sqm

Mail -> mailsend

Network -> Telephony -> asterisk13
asterisk13-app-directed_pickup
asterisk13-app-read
asterisk13-app-stack
asterisk13-app-system
asterisk13-cdr
asterisk13-cdr-csv
asterisk13-chan-sip
asterisk13-codec-a-mu
asterisk13-codec-alaw
asterisk13-codec-g722
asterisk13-codec-g726
asterisk13-codec-gsm
asterisk13-codec-ilbc
asterisk13-codec-resample
asterisk13-codec-ulaw
asterisk13-format-gsm
asterisk13-format-sln
asterisk13-func-blacklist
asterisk13-func-groupcount
asterisk13-pbx-spool
asterisk13-res-musiconhold

make

...

make[3] -C toolchain/musl prepare
make[3] -C toolchain/musl compile
make[3] -C toolchain/musl install
make[3] -C toolchain/gcc/final prepare
make[3] -C toolchain/gcc/final compile
make -r world: build failed. Please re-run make with -j1 V=s to see what's going on
make: *** [/home/dennis/Downloads/source-17.01.0-rc1/include/toplevel.mk:199: world] Fehler 1

make -j1 V=s

...

hain-mips_24kc_gcc-5.4.0_musl-1.1.15/gcc-5.4.0/missing makeinfo --split-size=5000000 --split-size=5000000" 'AR=ar' 'AS=as' 'CC=x86_64-pc-linux-gnu-gcc' 'CXX=x86_64-pc-linux-gnu-g++' 'DLLTOOL=dlltool' 'GCJ=' 'GFORTRAN=' 'GOC=' 'LD=ld' 'LIPO=lipo' 'NM=nm' 'OBJDUMP=objdump' 'RANLIB=ranlib' 'READELF=readelf' 'STRIP=strip' 'WINDRES=windres' 'WINDMC=windmc' LDFLAGS="${LDFLAGS}" HOST_LIBS="${HOST_LIBS}" "GCC_FOR_TARGET= $r/./gcc/xgcc -B$r/./gcc/" "echo 'STMP_FIXPROTO=' | sed -e s'/[^=][^=]*=$/XFOO=/'" "echo 'LIMITS_H_TEST=' | sed -e s'/[^=][^=]*=$/XFOO=/'" all
make[5]: *** [Makefile:4118: all-gcc] Error 139
make[5]: Leaving directory '/home/dennis/Downloads/source-17.01.0-rc1/build_dir/toolchain-mips_24kc_gcc-5.4.0_musl-1.1.15/gcc-5.4.0-final'
make[4]: *** [Makefile:874: all] Error 2
make[4]: Leaving directory '/home/dennis/Downloads/source-17.01.0-rc1/build_dir/toolchain-mips_24kc_gcc-5.4.0_musl-1.1.15/gcc-5.4.0-final'
make[3]: *** [Makefile:88: /home/dennis/Downloads/source-17.01.0-rc1/build_dir/toolchain-mips_24kc_gcc-5.4.0_musl-1.1.15/gcc-5.4.0-final/.built] Error 2
make[3]: Leaving directory '/home/dennis/Downloads/source-17.01.0-rc1/toolchain/gcc/final'
make[2]: *** [toolchain/Makefile:92: toolchain/gcc/final/compile] Error 2
make[2]: Leaving directory '/home/dennis/Downloads/source-17.01.0-rc1'
make[1]: *** [toolchain/Makefile:89: /home/dennis/Downloads/source-17.01.0-rc1/staging_dir/toolchain-mips_24kc_gcc-5.4.0_musl-1.1.15/stamp/.toolchain_install] Error 2
make[1]: Leaving directory '/home/dennis/Downloads/source-17.01.0-rc1'
make: *** [/home/dennis/Downloads/source-17.01.0-rc1/include/toplevel.mk:199: world] Fehler 2

See the .config file: https://www.file-upload.net/download-12292118/config.zip.html

Did you read the text above about SOURCE_DATE_EPOCH? I am not sure if that affects the toolchain build, but it might.

Make error 139 from gcc mean segmentation fault.
https://www.google.com/search?q=gcc+make+error+139
Something strange in your buildsystem? Normal recent 64bit Linux like Ubuntu 16.10 x64?

In any case, please discuss the compilation problems in a separate thread, not in this thread that explains how you can checkout the tagged release sources...

Something strange in your buildsystem? Normal recent 64bit Linux like Ubuntu 16.10 x64?

It is a Manjaro Linux (based on Arch Linux) and i can build the Image from https://github.com/lede-project/source

The tagging script has been modified for 17.01.0-rc2 to create and include "version.date" and "version" files along the sources. Those two files will provide the correct date and git version that will work also when the source has been fetched as .tar.gz or .zip archive instead of git.

Note to git users: the two files "version.date" and "version" in the buildroot root will also override the version and date from git commits. In practice that means that if the rc2 (and later) sources are fetched via git and then new private commits are added to repo, the build system will overlook those when calculating date & version, and will use the rc2 date and git version info for the build. Those two files can be deleted, and then the build will again use "last git commit" based date & version.

In any case, fetching rc1/rc2/release sources is meant to create sources exactly matching the release. If you are going to add stuff to the build, you are likely better off from fetching directly from the release branch's (like lede-17.01) HEAD and building from that, so that you get also all the later bug fixes.

Hi, how can I compile the final 17.01.0?
Thanks

Just following the advice in the first message of this thread. But adapting the tag name to the one that you want, this time v17.01.0

git clone https://git.lede-project.org/source.git
git fetch --tags
git tag -l
git checkout v17.01.0

(those commands close the git repo, fetch tags, list tags for you, finally you checkout the required one based on the tag list)

And then normal compile process with feed updating etc. and finally "make".
But advice for that is outside the scope of this thread. Look into wiki for that.

Many thanks :slight_smile:

Where? I can't find it...

You can start there:
https://lede-project.org/docs/guide-developer/build-system

Thanks again for your help but into wiki there isn't anything about

git fetch --tags
git tag -l
git checkout v17.01.0

:smiley:

@hnyman, does this still apply to the v17.01.0 head if you download the source as an archive?

I compiled a new build but I have the same issues, buildnumber is unknown and the base-files under software.
I will to re-compile using your command.

Loading final 17.01.0 as archive and building from it should work OK as @jow fixed the tagging by adding version & date files to the final tagged commit.

But I have not tested it myself, yet.

When downloading head 17.01 as an archive gives the same issue also with your command. With git all went fine.

The archive download and build only works for the release tags like 17.01.0.. downloading the head of 17.01 branch should be done with git.

@hnyman thank you for clearifying that.

The advice is valid for the 17.01.1 release:

$ git fetch --tags
remote: Counting objects: 42, done.
remote: Total 42 (delta 27), reused 27 (delta 27), pack-reused 15
Unpacking objects: 100% (42/42), done.
From https://github.com/lede-project/source
 * [new tag]               v17.01.0     -> v17.01.0
 * [new tag]               v17.01.0-rc1 -> v17.01.0-rc1
 * [new tag]               v17.01.0-rc2 -> v17.01.0-rc2
 * [new tag]               v17.01.1     -> v17.01.1

$ git checkout v17.01.1
Note: checking out 'v17.01.1'.

You are in 'detached HEAD' state. You can look around, make experimental
...
HEAD is now at 48461b5abc... LEDE v17.01.1: adjust config defaults

$ git log --oneline -n 4
48461b5abc LEDE v17.01.1: adjust config defaults
7eb58cf109 utils/f2fs-tools: Update to 1.8.0
1d76542cca busybox: add musl compatible nslookup replacement
6ca5ccc620 kernel: update kernel 4.4 to 4.4.61

Note:
Checking out the exact release locks also the feeds (packages, LuCI etc..) to the release moment. Later fixes made to LuCI and packages will not get compiled into the firmware. Similarly fixes made later to the 17.01 branch main sources do not get included.

For most users it is usually better to compile from the HEAD of the 17.01 branch instead of compiling the exact release like 17.01.1. When compiling from branch HEAD, the later fixes in the main 17.01 source and in the feeds get included into the build. To achieve that, do not checkout the release tag "v17.01.1" but checkout the branch "lede-17.01":
git checkout lede-17.01

which means that

git clone https://github.com/openwrt/openwrt.git
cd openwrt/
(git fetch --tags)
git checkout openwrt-18.06

is the same as

git clone -b openwrt-18.06 --single-branch https://git.openwrt.org/openwrt/openwrt.git

Right?

Checkout means checking out branches or tracking branches. This was confusing for me since the branch openwrt-18.06.1 is not "selectable" on the github web ui

And

git checkout openwrt-18.06

is not the same as

git checkout -b openwrt-18.06

Not quite. The files themselves are the same, but the git history is more limited:
The "clone single-branch" does not fetch the commit history for other branches like master, so it will be impossible to e.g. cherry-pick a later commit from master.

There is no branch 18.06.1
There is the 18.06 branch and the tag 18.06.1 pointing to the 18.06.1 commit in the 18.06 branch.
See the 18.06 branch changelog and notice the tag in the history: https://git.openwrt.org/?p=openwrt/openwrt.git;a=shortlog;h=4ffed014a36b3d3ed9d71227adb8158ac8fd7564