Saving dynamically-generated files during sysupgrade

I'm looking to get back to an idea from a couple of years ago, to save the current luci_statistics data automatically during shutdown/reboot, sysupgrade, and config file backups.

What I would need to do is:

  • During sysupgrade, run a script to shut down statistics gathering and save the volatile data into a flash-based location, and add that file to the preserved data list. There's a way to do this for shutdown/reboot, such as Save luci-statistics/collectd across reboot - #21 by user674574.
    I think getting this to run during sysupgrade is straightforward, by installing a new script file such as /lib/upgrade/xyz.sh which can run in stage 1 of sysupgrade to stop the services and save the volatile file contents and include the saved file in the backup list

  • During backup file creation (sysupgrade -b or the LuCi equivalent), dynamically generate the backup file for the statistics data. This one I'm unclear on how/if it's possible to do simply. My goal is that when I make a system config file backup, it generates a snapshot of the current statistics data and includes that in the backup. This might require adding a script callout mechanism to sysupgrade -b to run scripts before generating backups and another set after generating backups? If we had such callouts, then maybe I don't even need an upgrade script in /lib/upgrade, since the backup generation happening during a real sysupgrade would be able to run the callouts.

Any suggestions on where to head with this? I was hoping not to have to touch sysupgrade itself, and just be able to manage things with added scripts to be installed by packages that want to preserve data on-the-fly.

1 Like

Well, since you can augment what sysupgrade -b saves by just adding files to /etc/sysupgrade.conf would it be as easy as:

$ echo /etc/collectd.bk >> /etc/sysupgrade.conf   # Do this just once.

$ collectd_backup.sh > /etc/collectd.bk
$ sysupgrade blah blah blah
... reboots
$ collectd_restore.sh /etc/collectd.bk  # Put this in the first boot script for the upgrade???

Maybe I didn't explain properly? The file to be saved needs to be generated automatically at the time that sysupgrade runs. The data live at run-time in /tmp, I need sysupgrade to call my code (script) to stop the data collection and copy the data into a flash-persistent location (such as /etc/backup/rrdbackup.tgz), then include this file with all the other backup files into its backup. And if it's just running a backup and not actually running an upgrade, call again to external code (script) to resume the statistics gathering.

My goal is for this to happen anytime someone calls sysupgrade -b, or uses LuCi to download a backup, or runs sysupgrade to really upgrade the system.

Yes, I misunderstood. After reading the page you linked, all the heavy lifting is already done, we just need a way to automatically wire it into the reboot process.

What would be nice is if the underlyingreboot were to fire off a hotplug event that we could intercept and do cleanup tasks. I think all of sysupgrade, LuCI ASU and auc use ubus call rpc-sys reboot, so maybe there's something going on there we could take advantage of?

A "pre-backup" hook would look nice right before line 234. Roughly,

    for script in /etc/sysupgrade.d/* ; do
        $script
    done

Oh, but now I realize it's a shell script. So that's not as complicated to prototype a change as I thought (don't need a full toolchain setup). But, of course any changes to it demand careful review and testing.

1 Like

Hmm, actually that run_hooks may be sufficient.
I didn't realize that was run on every sysupgrade -b and also by a sysupgrade that is going to install a new release.

So inside any of the hooks installed by /lib/upgrade/*.sh, we can have a script that stops statistics gathering, saves the volatile files to /etc/somewhere, puts /etc/somewhere into the backup file list, and restarts statistics.
(Well, it could check config options about whether to do that, and also check if statistics are currently running.)

Also, add code to the shutdown part of /etc/init.d/luci_statistics to save the statistics

1 Like

That run_hooks code is 16 years old, Felix anticipated this use case. :+1:t3:

I tested by adding /lib/upgrade/00-test.sh with just a logger command to dump the environment, then ran sysupgrade -b /tmp/bork.tar.gz. There's some interesting stuff there for other applications.

CONFFILES=/tmp/sysupgrade.conffiles
CONF_BACKUP=/tmp/bork.tar.gz
CONF_BACKUP_LIST=0
CONF_IMAGE=
CONF_RESTORE=
CONF_TAR=/tmp/sysupgrade.tgz
ETCBACKUP_DIR=/etc/backup
FORCE=0
HELP=0
IGNORE_MINOR_COMPAT=0
INSTALLED_PACKAGES=/etc/backup/installed_packages.txt
MTD_ARGS=
MTD_CONFIG_ARGS=
NEED_IMAGE=1
SAVE_CONFIG=1
SAVE_INSTALLED_PKGS=0
SAVE_OVERLAY=0
SAVE_OVERLAY_PATH=
SAVE_PARTITIONS=1
UMOUNT_ETCBACKUP_DIR=0

Ok, that's pretty much what the rrdbackup script in your link above does.

It could use some fixes, like the one below (to pluck the actual rrd data location from the luci_statististics config). Most of it could just be merged into the already existing /etc/init.d/luci_statistics script, adding the new backup and restore commands to it, adding a stop_service and augmenting the start_service with a call to restore.

RRD_DIR=$(uci get luci_statistics.collectd_rrdtool.DataDir | sed 's;^/;;')

Then the /lib/upgrade/luci_statistics.sh file would be a one-liner.

#!/bin/sh
/etc/init.d/luci_statistics stop # 'stop' always calls 'backup'...

Oh, also need to tell sysupgrade to keep our new /etc/luci_statistics/ stuff. Might be easiest to just add another file, /lib/upgrade/keep.d/luci_statistics containing just that file:

/etc/luci_statistics/stats.tar.gz

Some thoughts:

  • LuCI statistics is piggybacking the underlying collectd, which actually handles all collection and database stuff. The backup / restore functionality might also be there.

  • If you go with sysupgrade hook method, depending on which sysupgrade hook you attach the call, the function might actually get run twice if you sysupgrade via LuCI. The image verification phase in LuCI first calls sysupgrade with "test" option, so that nothing is actually written but almost whole sysupgrade routine is still run. After the image has been checked and user presses the final ok, sysupgrade gets run for the second time for the actual action.

    So, any data collection stop/restart action needs to be carefully placed.

  • I haven't checked the backup vs. kill-services action order in sysupgrade. Just placing code in luci_statistics stop function might be too late, if backup has already been run before finally terminating the services.

I have avoided putting automatics for save/restore, as I flash test builds so often. Better for me to keep the sysupgrade automatics as minimal as possible (to easier notice the possible bugs in the main code).

I just have commands in place in Luci commands screen, so that I can do the save and restore manually from LuCI.

1 Like

I was thinking to make it an option (enable/disable) in the collectd_rrdtool section of luci_statistics whether to do the backup/restore.

The syupgrade script can't just stop luci_statistics, it needs to stop it, save data, and restart it, because running a backup (sysupgrade -b) will run all the sysupgrade scripts. As you say, we need to check what conditions are during backup and during a complete sysupgrade to determine the necessary actions.

Unrelated to this, I noticed that simply stopping luci_statistics doesn't stop collectd. Should it? Currently, only restarting luci_statistics will restart collectd.

root@LEDE:~# ps w |grep collectd
 6663 root      4568 SN   /usr/sbin/collectd -C /tmp/collectd.conf -f
32318 root      1100 S    grep collectd
root@LEDE:~# service luci_statistics stop
root@LEDE:~# ps w |grep collectd
 6663 root      4568 SN   /usr/sbin/collectd -C /tmp/collectd.conf -f
32345 root      1100 S    grep collectd
root@LEDE:~# service luci_statistics start
root@LEDE:~# ps w |grep collectd
 6663 root      4568 SN   /usr/sbin/collectd -C /tmp/collectd.conf -f
32398 root      1100 S    grep collectd
root@LEDE:~# service luci_statistics restart
root@LEDE:~# ps w |grep collectd
32481 root      4096 SN   /usr/sbin/collectd -C /tmp/collectd.conf -f
32528 root      1100 R    grep collectd
root@LEDE:~# 

Well, Luci statistics is merely a config generating script around collectd. There is nothing actually running.

Stopping stats should probably stop also collectd, although in theory there might also be other items active with collectd.

In my own backup script I directly stop/start collectd instead of LuCI stats (as it makes no sense to regenerate config for collected, as the previous config is already there.)

Looking around at how this is all assembled, here's my understanding (writing it down to check if I got it right): /etc/config/collectd by default identifies /etc/collectd.conf as an alternate config file. /etc/init.d/collectd looks for the alternate config file and, if found, uses it exclusively.

/etc/collectd.conf is not installed with collectd. /etc/init.d/luci_statistics takes control of /etc/collectd.conf when it starts. Once luci_statistics manages /etc/collectd.conf, there isn't a way to use collectd for other uses.

Another question about the data files stored by collectd on behalf of luci_statistics. Is it necessary to stop collectd to get a consistent backup from /tmp/rrd ( luci_statistics.collectd_rrdtool.DataDir)?
Or is it safe to just take a copy of the directory while collectd/rrdtool might be running?

You might take a copy while running, but there might be a write operation ongoing, as collectd plugins write data asynchronously.

So I have erred toward cautiousness and stopped collectd before doing the file copy.

edit:
Ps.
I am cautious regarding collectd, because it is also using external plugins for data generation, partially via callbacks.

Example of my early travails regarding collectd timings... https://github.com/collectd/collectd/issues/1227

1 Like

Yeah, my opinion is moving towards just logging on an external (non-volatile) drive as a solution to all this... (Easy for me to say as the two routers I care about are x86, so non-volatile, non-flash space is abundant.)

Is there a quick way to build the luci_statistics package? I'm looking at https://openwrt.org/docs/guide-developer/toolchain/single.package , which works but it does a bunch of prep work on every build. Can I do a different make invocation that skips all the prep work and just builds the package? Seems like I should be able to generate it in far less than 2m56s.

osboxes@osboxes:~/openwrt$ time make package/luci-app-statistics/compile |&tee -a BUILD.log
WARNING: Makefile 'package/feeds/packages/adblock-fast/Makefile' has a dependency on 'curl', which does not exist
WARNING: Makefile 'package/feeds/packages/afuse/Makefile' has a dependency on 'libfuse', which does not exist
WARNING: Makefile 'package/feeds/packages/afuse/Makefile' has a dependency on 'fuse-utils', which does not exist
WARNING: Makefile 'package/feeds/packages/aircrack-ng/Makefile' has a dependency on 'usbutils', which does not exist
WARNING: Makefile 'package/feeds/packages/apache/Makefile' has a dependency on 'libnghttp2', which does not exist
WARNING: Makefile 'package/feeds/packages/apache/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/apcupsd/Makefile' has a dependency on 'libusb-compat', which does not exist
WARNING: Makefile 'package/feeds/telephony/asterisk/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/avrdude/Makefile' has a dependency on 'libusb-compat', which does not exist
WARNING: Makefile 'package/feeds/packages/backuppc/Makefile' has a dependency on 'iputils-ping', which does not exist
WARNING: Makefile 'package/feeds/packages/bind/Makefile' has a dependency on 'libnghttp2', which does not exist
WARNING: Makefile 'package/feeds/packages/boinc/Makefile' has a dependency on 'curl', which does not exist
WARNING: Makefile 'package/feeds/packages/btrfs-progs/Makefile' has a dependency on 'liblzo', which does not exist
WARNING: Makefile 'package/feeds/packages/clamav/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/clamav/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/clocate/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/collectd/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/collectd/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/collectd/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/collectd/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/collectd/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/collectd/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/conntrack-tools/Makefile' has a dependency on 'libnetfilter-cttimeout', which does not exist
WARNING: Makefile 'package/feeds/packages/conntrack-tools/Makefile' has a dependency on 'libnetfilter-cthelper', which does not exist
WARNING: Makefile 'package/feeds/packages/conntrack-tools/Makefile' has a dependency on 'libnetfilter-queue', which does not exist
WARNING: Makefile 'package/feeds/packages/conntrack-tools/Makefile' has a dependency on 'libnetfilter-cttimeout', which does not exist
WARNING: Makefile 'package/feeds/packages/conntrack-tools/Makefile' has a dependency on 'libnetfilter-cthelper', which does not exist
WARNING: Makefile 'package/feeds/packages/conntrack-tools/Makefile' has a dependency on 'libnetfilter-queue', which does not exist
WARNING: Makefile 'package/feeds/packages/czmq/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/davfs2/Makefile' has a dependency on 'libfuse', which does not exist
WARNING: Makefile 'package/feeds/packages/ddns-scripts/Makefile' has a dependency on 'curl', which does not exist
WARNING: Makefile 'package/feeds/packages/ddns-scripts/Makefile' has a dependency on 'curl', which does not exist
WARNING: Makefile 'package/feeds/packages/ddns-scripts/Makefile' has a dependency on 'curl', which does not exist
WARNING: Makefile 'package/feeds/packages/ddns-scripts/Makefile' has a dependency on 'curl', which does not exist
WARNING: Makefile 'package/feeds/packages/ddns-scripts/Makefile' has a dependency on 'curl', which does not exist
WARNING: Makefile 'package/feeds/packages/ddns-scripts/Makefile' has a dependency on 'curl', which does not exist
WARNING: Makefile 'package/feeds/packages/ddns-scripts/Makefile' has a dependency on 'curl', which does not exist
WARNING: Makefile 'package/feeds/packages/ddns-scripts/Makefile' has a dependency on 'curl', which does not exist
WARNING: Makefile 'package/feeds/packages/ddns-scripts/Makefile' has a dependency on 'curl', which does not exist
WARNING: Makefile 'package/feeds/packages/ddns-scripts/Makefile' has a dependency on 'curl', which does not exist
WARNING: Makefile 'package/feeds/packages/ddns-scripts/Makefile' has a dependency on 'curl', which does not exist
WARNING: Makefile 'package/feeds/packages/ddns-scripts/Makefile' has a dependency on 'curl', which does not exist
WARNING: Makefile 'package/feeds/packages/ddns-scripts/Makefile' has a dependency on 'curl', which does not exist
WARNING: Makefile 'package/feeds/packages/ddns-scripts/Makefile' has a dependency on 'curl', which does not exist
WARNING: Makefile 'package/feeds/packages/digitemp/Makefile' has a dependency on 'libusb-compat', which does not exist
WARNING: Makefile 'package/feeds/packages/dnsdist/Makefile' has a dependency on 'libnghttp2', which does not exist
WARNING: Makefile 'package/feeds/packages/dnsdist/Makefile' has a dependency on 'libnghttp2', which does not exist
WARNING: Makefile 'package/feeds/packages/domoticz/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/elektra/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/eoip/Makefile' has a dependency on 'liblzo', which does not exist
WARNING: Makefile 'package/feeds/packages/esniper/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/etherwake-nfqueue/Makefile' has a dependency on 'libnetfilter-queue', which does not exist
WARNING: Makefile 'package/feeds/packages/fdm/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/packages/freeradius3/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/packages/freeradius3/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/telephony/freeswitch/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/telephony/freeswitch/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/packages/fwknop/Makefile' has a dependency on 'libnetfilter-queue', which does not exist
WARNING: Makefile 'package/feeds/packages/gammu/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/git/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/gnunet-fuse/Makefile' has a dependency on 'libfuse', which does not exist
WARNING: Makefile 'package/feeds/packages/gnupg/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/gnupg2/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/haproxy/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/packages/haproxy/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/packages/hcxtools/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/hs20/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/https-dns-proxy/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/hub-ctrl/Makefile' has a dependency on 'libusb-compat', which does not exist
WARNING: Makefile 'package/feeds/packages/icecast/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/idevicerestore/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/telephony/kamailio/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/telephony/kamailio/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/telephony/kamailio/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/telephony/kamailio/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/telephony/kamailio/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/telephony/kamailio/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/telephony/kamailio/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/telephony/kamailio/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/knot-resolver/Makefile' has a dependency on 'libnghttp2', which does not exist
WARNING: Makefile 'package/feeds/packages/lcd4linux/Makefile' has a dependency on 'libusb-compat', which does not exist
WARNING: Makefile 'package/feeds/packages/lcd4linux/Makefile' has a dependency on 'libusb-compat', which does not exist
WARNING: Makefile 'package/feeds/packages/lcdproc/Makefile' has a dependency on 'libusb-compat', which does not exist
WARNING: Makefile 'package/feeds/packages/leech/Makefile' has a dependency on 'curl', which does not exist
WARNING: Makefile 'package/feeds/packages/libcurl-gnutls/Makefile' has a build dependency on 'curl', which does not exist
WARNING: Makefile 'package/feeds/packages/libdaq3/Makefile' has a dependency on 'libnetfilter-queue', which does not exist
WARNING: Makefile 'package/feeds/packages/libftdi/Makefile' has a dependency on 'libusb-compat', which does not exist
WARNING: Makefile 'package/feeds/packages/libgphoto2/Makefile' has a dependency on 'libusb-compat', which does not exist
WARNING: Makefile 'package/feeds/packages/libgphoto2/Makefile' has a dependency on 'libusb-compat', which does not exist
WARNING: Makefile 'package/feeds/packages/libmariadb/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/libmariadb/Makefile' has a build dependency on 'curl', which does not exist
WARNING: Makefile 'package/feeds/packages/libndpi/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/packages/libnpupnp/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/libsoup3/Makefile' has a dependency on 'libnghttp2', which does not exist
WARNING: Makefile 'package/feeds/packages/libulfius/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/libulfius/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/linknx/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/lua-curl-v3/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/luarocks/Makefile' has a dependency on 'curl', which does not exist
WARNING: Makefile 'package/feeds/packages/mocp/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/mpd/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/mpd/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/mpd/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/routing/naywatch/Makefile' has a dependency on 'owipcalc', which does not exist
WARNING: Makefile 'package/feeds/packages/netifyd/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/netopeer2/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/nmap/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/packages/nmap/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/packages/nmap/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/packages/node/Makefile' has a dependency on 'libnghttp2', which does not exist
WARNING: Makefile 'package/feeds/packages/ntfs-3g/Makefile' has a dependency on 'libfuse', which does not exist
WARNING: Makefile 'package/feeds/packages/nut/Makefile' has a dependency on 'libusb-compat', which does not exist
WARNING: Makefile 'package/feeds/packages/openthread-br/Makefile' has a dependency on 'libnetfilter-queue', which does not exist
WARNING: Makefile 'package/feeds/packages/openwisp-config/Makefile' has a dependency on 'curl', which does not exist
WARNING: Makefile 'package/feeds/packages/owfs/Makefile' has a dependency on 'libfuse', which does not exist
WARNING: Makefile 'package/feeds/packages/owfs/Makefile' has a dependency on 'fuse-utils', which does not exist
WARNING: Makefile 'package/feeds/packages/owfs/Makefile' has a dependency on 'libusb-compat', which does not exist
WARNING: Makefile 'package/feeds/packages/owntone/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/pdns/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/pdns/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/pdns-recursor/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/perl-device-usb/Makefile' has a dependency on 'libusb-compat', which does not exist
WARNING: Makefile 'package/feeds/packages/perl-www-curl/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/php8/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/php8-pecl-http/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/postfix/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/packages/privoxy/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/packages/python-curl/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/python3-iperf3/Makefile' has a dependency on 'libiperf3', which does not exist
WARNING: Makefile 'package/feeds/packages/rclone/Makefile' has a dependency on 'fuse-utils', which does not exist
WARNING: Makefile 'package/feeds/packages/rsyslog/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/rsyslog/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/rtorrent/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/rtorrent/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/telephony/rtpengine/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/telephony/rtpengine/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/packages/serdisplib/Makefile' has a build dependency on 'libusb-compat', which does not exist
WARNING: Makefile 'package/feeds/packages/shairport-sync/Makefile' has a dependency on 'libconfig', which does not exist
WARNING: Makefile 'package/feeds/packages/shairport-sync/Makefile' has a dependency on 'libconfig', which does not exist
WARNING: Makefile 'package/feeds/packages/shairport-sync/Makefile' has a dependency on 'libconfig', which does not exist
WARNING: Makefile 'package/feeds/telephony/sipgrep/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/packages/sispmctl/Makefile' has a dependency on 'libusb-compat', which does not exist
WARNING: Makefile 'package/feeds/packages/snort/Makefile' has a dependency on 'libnghttp2', which does not exist
WARNING: Makefile 'package/feeds/packages/snort/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/packages/snort3/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/packages/speedtestcpp/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/squashfs-tools/Makefile' has a dependency on 'liblzo', which does not exist
WARNING: Makefile 'package/feeds/packages/squashfs-tools/Makefile' has a dependency on 'liblzo', which does not exist
WARNING: Makefile 'package/feeds/packages/sslh/Makefile' has a dependency on 'libconfig', which does not exist
WARNING: Makefile 'package/feeds/packages/strongswan/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/syslog-ng/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/tinc/Makefile' has a dependency on 'liblzo', which does not exist
WARNING: Makefile 'package/feeds/packages/tinc/Makefile' has a build dependency on 'lzo', which does not exist
WARNING: Makefile 'package/feeds/packages/transmission/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/transmission/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/transmission/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/travelmate/Makefile' has a dependency on 'curl', which does not exist
WARNING: Makefile 'package/feeds/packages/tvheadend/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/packages/uacme/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/ulogd/Makefile' has a dependency on 'libnetfilter-log', which does not exist
WARNING: Makefile 'package/feeds/packages/ulogd/Makefile' has a dependency on 'libnetfilter-log', which does not exist
WARNING: Makefile 'package/feeds/packages/ulogd/Makefile' has a build dependency on 'libnetfilter-log', which does not exist
WARNING: Makefile 'package/feeds/packages/umurmur/Makefile' has a dependency on 'libconfig', which does not exist
WARNING: Makefile 'package/feeds/packages/umurmur/Makefile' has a dependency on 'libconfig', which does not exist
WARNING: Makefile 'package/feeds/packages/unbound/Makefile' has a dependency on 'libnghttp2', which does not exist
WARNING: Makefile 'package/feeds/packages/wg-installer/Makefile' has a dependency on 'owipcalc', which does not exist
WARNING: Makefile 'package/feeds/packages/wg-installer/Makefile' has a dependency on 'owipcalc', which does not exist
WARNING: Makefile 'package/feeds/packages/wg-installer/Makefile' has a dependency on 'curl', which does not exist
WARNING: Makefile 'package/feeds/packages/xmlrpc-c/Makefile' has a dependency on 'libcurl', which does not exist
WARNING: Makefile 'package/feeds/packages/ykclient/Makefile' has a dependency on 'curl', which does not exist
WARNING: Makefile 'package/feeds/packages/ykpers/Makefile' has a dependency on 'curl', which does not exist
WARNING: Makefile 'package/feeds/packages/yubico-pam/Makefile' has a dependency on 'curl', which does not exist
WARNING: Makefile 'package/feeds/packages/zabbix/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/packages/zabbix/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/packages/zabbix/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/packages/zabbix/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/packages/zabbix/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/packages/zabbix/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/packages/zabbix/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/packages/zabbix/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/packages/zabbix/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/packages/zabbix/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/packages/zabbix/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/packages/zabbix/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/packages/zabbix/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/packages/zabbix/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/packages/zabbix/Makefile' has a dependency on 'libpcre', which does not exist
WARNING: Makefile 'package/feeds/packages/zabbix/Makefile' has a dependency on 'libpcre', which does not exist
make[2]: Entering directory '/home/osboxes/openwrt/scripts/config'
make[2]: 'conf' is up to date.
make[2]: Leaving directory '/home/osboxes/openwrt/scripts/config'
 make[1] package/luci-app-statistics/compile
 make[2] -C /home/osboxes/luci/modules/luci-base host-compile
 make[2] -C package/libs/toolchain compile
 make[2] -C package/utils/lua compile
 make[2] -C package/libs/libjson-c compile
 make[2] -C package/libs/libnl-tiny compile
 make[2] -C package/libs/libubox compile
 make[2] -C package/system/ubus compile
 make[2] -C package/system/uci compile
 make[2] -C package/utils/ucode compile
 make[2] -C /home/osboxes/luci/contrib/package/lucihttp compile
 make[2] -C package/network/utils/iwinfo compile
 make[2] -C package/system/rpcd compile
 make[2] -C /home/osboxes/luci/libs/rpcd-mod-luci compile
 make[2] -C /home/osboxes/luci/contrib/package/ucode-mod-html compile
 make[2] -C /home/osboxes/packages/net/cgi-io compile
 make[2] -C package/utils/lua host-compile
 make[2] -C /home/osboxes/luci/contrib/package/csstidy host-compile
 make[2] -C /home/osboxes/luci/modules/luci-base clean-build
 make[2] -C /home/osboxes/luci/modules/luci-base compile
 make[2] -C package/libs/libtool compile
 make[2] -C package/libs/zlib compile
 make[2] -C package/firmware/linux-firmware compile
 make[2] -C package/kernel/gpio-button-hotplug compile
 make[2] -C package/kernel/linux compile
 make[2] -C package/libs/libmnl compile
 make[2] -C package/libs/libnftnl compile
 make[2] -C package/network/utils/iptables compile
 make[2] -C package/libs/ncurses host-compile
 make[2] -C package/libs/ncurses compile
 make[2] -C /home/osboxes/packages/libs/liboping compile
 make[2] -C /home/osboxes/packages/utils/rrdtool1 compile
 make[2] -C /home/osboxes/packages/utils/collectd compile
 make[2] -C /home/osboxes/luci/applications/luci-app-statistics compile

real	2m56.460s
user	2m24.123s
sys	0m44.993s

(edited to correct time summary)

Just don't do make clean for the whole build, but just for LuCI stats, so the next build has all dependencies already there as built. But you need to build them once.

make package/luci-app-statistics/clean

All dependencies are needed, as the source code might have links to library functions in them. LuCI stats doesn't really have those, but the make system doesn't know that in advance. (Main dependency collectd has especially many dependencies, as its plugins link to lots of low-level libraries. Keeping the selected collected plugins in minimum helps)

Ps. Wasn't that 2m56s, not 2h56m?

Whoops, yup! Still longer than I'd like for a quick "compile"/install/test cycle but I'll just have to live with it.

Pull request opened for an implementation.