Update to newest odhcpd on LEDE

treysis · March 21, 2019, 2:27pm

Hey everyone,

I am currently running LEDE 17.01.6 on an WR841NDv7. I can't upgrade to 18 as space isn't enough (maybe if I compile everything into the image it might work).

The problem is that the LEDE-included version of odhcpd doesn't satisfy my needs. It is missing the "ra_dns" switch, which was added only on Dec 31, 2018:

Is it possible to compile a newer version of odhcpd for my LEDE build? Would I just have to apply the patch to the sources? Or can I somehow just compile the package and update via opkg?

I have read this:
https://openwrt.org/docs/guide-developer/single.package
But I don't understand how to integrate the source for the latest odhcpd version/snapshot into the LEDE build system?

mikma · March 21, 2019, 2:40pm

I don't know if it's possible to recompile a recent odhcpd version for LEDE. If the patch apply cleanly to the version you use then it may be easiest.
You would in this case download the lede-sdk, and copy odhcpd from the lede source to the package/ subdirectory. Put the patch in the odhcpd/patches/ directory. Then you need to install package dependencies from the feeds, and build.

https://openwrt.org/docs/guide-developer/obtain.firmware.sdk
https://openwrt.org/docs/guide-developer/using_the_sdk

jeff · March 21, 2019, 3:13pm

As you're finding, 4/32 devices are past the point of "turn-key" support. They've got perhaps another year or two of "custom" support, provided by users to meet their own specific needs. I'd start considering purchasing forward-looking hardware as you keep your device running.

There are so many versions of the 841 series, but several are supported on the "tiny" target under 18.06.

Learning to use the image builder or build system will let you get a little more life out of the device. These will let you, for example:

Remove the few kernel modules you don't need (small gain)
Remove LuCI (significant gain)
Put packages you want into the ROM (roughly halves the flash consumption for packages over installing in overlay)

treysis · March 21, 2019, 7:11pm

@mikma
Isn't it somehow possible to build the most current odhcpd from git and then install the pkg-file? Or would that require to set up the whole build system only for odhcpd? Is it maybe possible to integrate the git source into the build system of 17.01.6?

@jeff
I know. But that doesn't help me pointing to a different device. This might even be only a temporary solution. But the question is interesting to me in general. Especially, since it seems that even the most recent 18 stable doesn't include the patched odhcpd. So my question would be valid for OpenWRT-18 as well.
I already stripped the device down to the bare necessities for my case, except for packing them into ROM. However, still odhcpd isn't the most current version.

jeff · March 21, 2019, 7:33pm

If you need to modify source, then you'll need to install the build system. While the SDK can do it as well, there is a lot of loss of flexibility compared to the full build system. I'd install and activate ccache to speed re-build iterations. If you're not familiar with git basics, those are important so that you can keep your changes current. If you're modifying upstream sources; Linux itself or package source, then the quick into to quilt on the wiki is needed as well.

git diff 1ca69003fd^ 1ca69003fd tells me that it is just a change in the Makefile, so that should be straightforward. git diff openwrt/openwrt-18.06 openwrt/master -- package/network/services/odhcpd/Makefile shows a subtle change in addition, that may or may not be important to pick up if you work from openwrt-18.06 branch.

treysis · March 21, 2019, 8:32pm

Ok. I still haven't grasped the whole git concept yet...I think I already installed the full build system (I thought that's what is called sdk? I downloaded using 'git clone -b 17.01.6 https://git.openwrt.org/openwrt/openwrt.git' - btw.: how can I see the current version of a git branch...the folder will only be called 'openwrt' and I don't find any version information inside it).

Can't I adjust the makefile so that it downloads the odhcpd-source from another git-branch (snapshot, or even the odhcpd project git)?

'git diff openwrt/openwrt-17.01.6 openwrt/master -- package/network/services/odhcpd/Makefile' throws an error: "fatal: bad revision 'openwrt/openwrt-17.01.6'".

jeff · March 21, 2019, 8:48pm

OK, easy one first -- as I understand the "SDK" it is a pre-compiled build chain for a specific version that saves somewhere between 15-45 minutes for a "first-time" build.

git takes some time to wrap your head around. It's, well, different than VCS like svn that have "heavy" branches and a fixed history. I guess one way to think of it is that git cares a lot about where you are now, not so much how you got there.

I don't know a good git tutorial, since it's been ages since I first learned it.

The key concepts to learn are probably those behind the commands

checkout
branch
add
commit
log
revert
cherry-pick
fetch
pull
rebase
bisect (just remember it exists for the day in the future that you need to figure which commit broke something)

Since you say that you can fit 18.06 onto the device, I'd do something like (as an example of what to learn, not what to follow blindly)

git remote rename origin openwrt    # Since I use multiple "upstreams", clearly identify this one
git checkout -b openwrt-18.06       # Start with the openwrt-18.06 branch and switch to a local copy
git checkout -b me-devel            # Create a local working branch that is the same as where we were (openwrt-18.06 HEAD)
git cherry-pick 1ca69003fd          # Grab and apply the single commit that makes the change
make menuconfig
make -j12 clean download world

You've got "all" of OpenWrt with a clone command. You can, for examples:

git log openwrt/master
git log openwrt/openwrt-18.06
git merge-base openwrt/openwrt-18.06 openwrt/master    # What is the "nearest" shared commit?
git log --pretty=oneline openwrt/master ^openwrt/openwrt-18.06  # What is on master that isn't on 18.06?

treysis · March 21, 2019, 9:14pm

I don't know if I can fit 18.06. At least compiling the SNAPSHOT (I think that's what I get if I don't specify the -b option with git clone?) with everything built in didn't produce an error.

Puh, that all still looks Greek to me...cherry-picking always throws bad-revision-error when submitting this commit "5e99738" as mentioned in the link above. I guess I'll have something ahead of me trying to understand git first.

I guess just downloading the latest .ipk won't work as probably it's built against some other version of ulibc and similar?

EDIT:
Isn't it possible to like integrate the changes of https://git.openwrt.org/?p=project/odhcpd.git from openwrt-18.06 branch into lede-17.01 branch?

slh · March 21, 2019, 10:04pm

What answer other than 'maybe' do you expect here?

From a git standpoint, yes - but changes in current odhcpd git HEAD may depend on changes or new features of its dependencies (e.g. netifd, dnsmasq, etc.) to function. This may include failing to build on 17.01.x, failing to run - or more subtle problems at runtime - or it might just work.

jeff · March 21, 2019, 10:16pm

A couple of things to help, I hope:

My openwrt/master or the like would likely be origin/master for you, until you git remote rename it to openwrt. The notation there is <remote name>/<branch name>

Here's how I found the latest commit that changed odhcpd:

Make sure I'm on the master branch (so I don't have to add openwrt/master to all the commands)
Give me the three latest commits related to package/network/services/odhcpd/Makefile
What files did it change?
Since not too many, what were the changes?

jeff@deb-devel:~/devel/openwrt$ git checkout master
Already on 'master'
Your branch is up-to-date with 'openwrt/master'.

jeff@deb-devel:~/devel/openwrt$ git log -3 --pretty=oneline -- package/network/services/odhcpd/Makefile
1ca69003fd odhcpd: update to latest git HEAD (FS#2160)
c8153722a2 odhcpd: update to latest git HEAD
0b4b1027c6 odhcpd: update to latest git HEAD (FS#2142)

jeff@deb-devel:~/devel/openwrt$ git diff --name-status 1ca69003fd^ 1ca69003fd
M       package/network/services/odhcpd/Makefile

jeff@deb-devel:~/devel/openwrt$ git diff 1ca69003fd^ 1ca69003fd
diff --git a/package/network/services/odhcpd/Makefile b/package/network/services/odhcpd/Makefile
index 45e13a452b..be54a41e12 100644
--- a/package/network/services/odhcpd/Makefile
+++ b/package/network/services/odhcpd/Makefile
@@ -12,9 +12,9 @@ PKG_RELEASE:=3
 
 PKG_SOURCE_PROTO:=git
 PKG_SOURCE_URL=$(PROJECT_GIT)/project/odhcpd.git
-PKG_SOURCE_DATE:=2019-02-27
-PKG_SOURCE_VERSION:=16c5b6c9bcb71bb67a6ab15bb458bf9ab5a8fe7e
-PKG_MIRROR_HASH:=bb42751da2a9d8fa1576db97524711af3fbf3758598f0dc631c075f6413796fd
+PKG_SOURCE_DATE:=2019-03-21
+PKG_SOURCE_VERSION:=6d23385242c918b0e00f5e21ed41dd655905752b
+PKG_MIRROR_HASH:=56e8fe3a3ec02371834c2b3af2c5b9e4106750b12b879b8334a95a40cfc4c5f8
 
 PKG_MAINTAINER:=Hans Dedecker <dedeckeh@gmail.com>
 PKG_LICENSE:=GPL-2.0

Now that hash, 1ca69003fd (I intentionally shorten them from default), may or may not apply "cleanly" as a cherry-pick to openwrt-18.06. I see that there were other commits prior, which may have been intermediate steps. If it didn't apply cleanly, I could either, while with my own development branch checked out:

Edit that file on my own branch, changing the three lines to the new (+) value, or
"Slam in" the version from master with git checkout openwrt/master package/network/services/odhcpd/Makefile, git add it, and git commit it to your own branch

That second one is a special case that says "get me that file from openwrt/master, rather than, without a path, "switch me over to the openwrt/master branch" Yes, it can be confusing that openwrt/master is different than master on your own machine.

Edit: "Slam in", as tempting as it might be, often breaks things. Note the subtle change near the end, when comparing with openwrt-18.06. Will it break something? It would be a guess for me.

There's also the change in PKG_VERSION that would need to be understood.

Anytime someone tells you "Oh, just backport it", now you have a brief taste of the issues.

jeff@deb-devel:~/devel/openwrt$ git diff openwrt/openwrt-18.06 package/network/services/odhcpd/Makefile
diff --git a/package/network/services/odhcpd/Makefile b/package/network/services/odhcpd/Makefile
index 34734c6e42..be54a41e12 100644
--- a/package/network/services/odhcpd/Makefile
+++ b/package/network/services/odhcpd/Makefile
@@ -9,18 +9,17 @@ include $(TOPDIR)/rules.mk
 
 PKG_NAME:=odhcpd
 PKG_RELEASE:=3
-PKG_VERSION:=1.15
 
 PKG_SOURCE_PROTO:=git
 PKG_SOURCE_URL=$(PROJECT_GIT)/project/odhcpd.git
-PKG_SOURCE_DATE:=2019-01-16
-PKG_SOURCE_VERSION:=ae16950ce18b037a91b706edcee92ce173cf74b0
-PKG_MIRROR_HASH:=ba6e1d7c5e3c26ca4e5c1d09077845033395656690be3faec21ed77c9bfca895
+PKG_SOURCE_DATE:=2019-03-21
+PKG_SOURCE_VERSION:=6d23385242c918b0e00f5e21ed41dd655905752b
+PKG_MIRROR_HASH:=56e8fe3a3ec02371834c2b3af2c5b9e4106750b12b879b8334a95a40cfc4c5f8
 
 PKG_MAINTAINER:=Hans Dedecker <dedeckeh@gmail.com>
 PKG_LICENSE:=GPL-2.0
 
-PKG_BUILD_DIR:=$(BUILD_DIR)/$(PKG_NAME)-$(BUILD_VARIANT)/$(PKG_NAME)-$(PKG_VERSION)
+PKG_BUILD_DIR=$(BUILD_DIR)/$(PKG_NAME)-$(BUILD_VARIANT)/$(PKG_NAME)-$(PKG_VERSION)
 
 PKG_INSTALL:=1
 PKG_CONFIG_DEPENDS:=CONFIG_PACKAGE_odhcpd_$(BUILD_VARIANT)_ext_cer_id

treysis · March 21, 2019, 11:37pm

Thank you. That was really helpful in understanding the basics! I feel much more comfortable with git now

I do understand the problems as well. The makefile-syntax basically has changed, so it might need adjustment of the according build scripts as well. Other things that could change would be other dependencies etc. etc. And yes, it failed compiling.

Now I also tried the different approach in that I just copied the sources over. That also made sense as the package name changed from odhcpd to odhcpd-ipv6only. Now luckily all dependencies still are met by 17.01.6 and I was able to compile the odhcpd-package, wget it onto my box and install it - it runs! And disabling ra_dns also works! But I have learned a lot here and feel more content to make my own builds. Probably I'll try to minify the newest snapshot for my system.

So thank you very much at this point!

treysis · March 22, 2019, 11:23pm

So I have a follow-up question:

Since i copied the sources of odhcpd to the 17.01.6 branch directory, why doesn't git tell me that the files changed? I mean, shouldn't git mention that I should commit my changes, or at least that there are changed files? Or doesn't it monitor odhcpd as this project has its own project page in git and is just imported during build time?

jeff · March 22, 2019, 11:54pm

There are several git "roots" or "trees" in the OpenWrt build system. The upper level doesn't track changes in the lower levels.

jeff@deb-devel:~/devel/openwrt$ find . -type d -name .git
./.git
./feeds/telephony/.git
./feeds/packages/.git
./feeds/luci/.git
./feeds/routing/.git
./env/.git

Also, the downloaded source is not tracked at all. It is downloaded based on the package definition, expanded, and patched at build time.

From master, just as an example

PKG_NAME:=odhcpd
PKG_RELEASE:=3

PKG_SOURCE_PROTO:=git
PKG_SOURCE_URL=$(PROJECT_GIT)/project/odhcpd.git
PKG_SOURCE_DATE:=2019-03-21
PKG_SOURCE_VERSION:=6d23385242c918b0e00f5e21ed41dd655905752b
PKG_MIRROR_HASH:=56e8fe3a3ec02371834c2b3af2c5b9e4106750b12b879b8334a95a40cfc4c5f8

treysis · March 23, 2019, 1:21pm

Alright. So let me write down my idea:

If I adjust the Makefile for odhcpd so that the hash in PKG_SOURCE_VERSION reflects a more current hash, then it will download this newer sources when I issue make? (and build successfully, assuming the dependencies are still met)

jeff · March 23, 2019, 2:12pm

While there's some "style" questions around if you modify the existing one in-place (on your own git branch, so you can later rebase your changes onto updated OpenWrt source) or make your own, yes, that probably will work for you. I'd bump the version up from whatever it is now, perhaps with a _1 suffix to denote your own versions. As I vaguely remember, the build logs will show the hash obtained by the download if the hash is missing. I build with developer options "logs on" and with ccache, though V=s probably shows it as well.

You'll also want to check the contents of files/ and patches/ to make sure they make sense for the different version, probably updating them as well as Makefile

treysis · August 8, 2019, 11:55am

In the end it worked. I used a more recent commit hash in PKG_SOURCE_VERSION and thus was able to build an image with a more recent odhcpd that includes ra_dns (15-May-2019). Tried the same on Chaos Calmer, but make didn't complete in that case. I'm happy now the way it is. Had an uptime of 132 days now without any issues.

system · August 18, 2019, 11:55am

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.