Since I upgraded from 19.07.3 to 19.07.6, some packages get completely rebuilt with every make, although nothing has changed in their source repo or makefile. I'm posting this before starting to dig deep into it - maybe there is a known change (unknown to me…) that might trigger package rebuilds differently than in 19.07.3?
I think the reason is not the build itself, but as it seems package sources get re-downloaded into /dl from the originating git repo, for reasons I do not yet understand (no changes in the repo, no changes in the project makefile).
I frequently switch the target in .config. This definitely did not trigger re-downloading packages already built for the respective target in 19.07.3.
I'm not sure yet if not switching targets would prevent unexpected re-downloads of packages now, but I just mention this in case there's a new dependency on target switches that was not present in 19.07.3.
I can answer my question now: Nov 19th, 2020, the following commit was made:
4e19cbc - download: handle possibly invalid local tarballs
If I understand that correctly, it forces running download.pl every time even if the tarball already existed. download.pl then checks if the PKG_HASH hash matches the source that is already there. If it does, the source will not be re-downloaded, so the tarball and the build directory will be left untouched.
Now, the problem with this in my case is that those packages I was working with, because under active development, don't have a PKG_HASH, and therefore could never match, so were downloaded every single time.
Futhermore, this patch also breaks the USE_SOURCE_DIR mechanism. Actually, only realizing this made me look into things - I find USE_SOURCE_DIR indispensable while developing my own packages.
I'll try to find a clean way to fix this and will post a patch.
[Update] Digging further, I realized download.pl is only involved when the source code is actually a tarball upstream. For sources cloned from git repositories (PKG_SOURCE_PROTO=git), the line
$(DL_DIR)/$(FILE): FORCE
introduced by the patch in package.mk simply causes a complete package rebuild every time.
I guess this line should be made conditional somehow, but I am not yet sure on what exactly.
Since the commit mentioned above, which makes sure no invalid local tarballs are accidentally used, a package without a PKG_HASH will be re-downloaded and re-built every time.
This is the correct and safe way to handle this.
But for the cases where this behaviour is unwanted (developing package with USE_SOURCE_DIR active) there is an easy workaround: