Update rrdtool up to 1.4 or higher

Hi!

I have seem a topic on version 1.2, but it is 3 years old, and an older version, and the author didn't publish his results.

I need to update rrdtool up to 1.4 to get the --json data dump in rrdttool.

I would like to use openwrt SDK, but from 1.0.50 version there are a lot of changes in the dependencies.

Any experience updating rrdtool up to 1.4 or higher version?

Any guide, tutorial, support or similar process with other openwrt package to follow it?

Thank you!

Work on the source and build the whole firmware image including your changes from source. The SDK only works (reasonably) for (modified) leaf packages, not libraries used by multiple other packages.

1 Like

I did the rrdtool upgrade a year ago for testing purposes, but did not keep it, as newer rrdtool versions require several nonstandard graphics/font libraries.

We have kept using the really ancient rrdtool 1.0.0.50 from year 2003/2005 as it is small and suitable for limited flash space.

(I added the cairo and pango libraries manually, but nowadays they exist also in the nonstandard video feed.)

Edit:
I upgraded to 1.7.2, and one example graph is here:

I could try digging up the source changes on weekend, when I am back home.

I have been checking the forum post. My target system is a misp 32 bits router, but I could try to readapt the code.

I really appreciate if you could find the source code for the rrdtool package with your changes.

Thank you @hnyman !

Btw, do you intend to use rrdtool as a separate tool, or to integrate it to LuCI statistics and collectd?

I am currently trying to compile the new version, and as I want to seamlessly integrate it into the LuCI statistics, there are a few more steps than for a pure separate tool installation.

Also note that the old rrdtool 1.0.50 has been patched to ignore the 64 bit time_t in 32 systems and to handle time data as native 32bit in 32bit systems.

I am not going to patch 1.7.2 that way, as it should recognise the time_t type better in any case. But the drawback from that is that the earlier databases from earlier rrdtool versions are not compatible (at least for some tools). That should not bother you if you use rrdtool separately, but could hurt you if you aim to use it to export old LuCI databases.

Hi @hnyman,

OK, I will describe my current setup and how I expect to use rrdtool. I think that I am going to be in troubles.

I am using opemwrt collectd and the mqtt plugin versions to collect sensors and home devices data into rrd databases.

I would like to use rrdtool to fetch data from these rrd databases an dump it using json format, with the rrddrv server. That way I will be able to draw data in Grafana.

I think, if I understand properly what you explained, that I am not going to be able to use rrdtool 1.7 to fetch data from the old format rrd databases generated by collectd plugin.

Is this the problem?

Thank you

Yes, it is.
You would need to patch 1.7.2 similarly as I finally did for 1.0.50 in the commit below, in order to be able to read from the old databases. 1.7.2 properly evaluates the size on time_t and recognises the 64bit time_t (which needs to be avoided)

I originally thought to go to the other direction, patching the ancient rrdtool to properly use 64bit time_t also in 32bit systems, but after discussing with jow, I went with the approach of keeping 32bit time_t in 32bit systems so that old databases are compatible.

Ps.
I compiled the 1.7.2 but it does not work, yet.
I think that the reason is that this time I used cairo, pango and mesa from the video feed, and they enable more features than necessary for the simple graphs. That leads into some runtime library linking problem. (and also caused my firmware image to grow by 4 MB)

Sadly I do not have the cairo & pango patches from the 2021 trial, when I created simple versions of the pango and cairo packages myself.

FYI, although you might not want this any more (due to the possible incompatibility with existing databases), I document here that I re-implemented support for new rrdtool versions.

Now there is already rrdtool 1.8.0.

I have only tested this in my mt7622/RT3200, which is a 64bit system, so the old databases created by 1.0.50 are compatible with the new rrdtool 1.8.0.

But with 32 bit systems, the time_t size is different and the old databases are incompatible. (but I haven't tested that yet with my R7800. Nor have I tried to create a compatibility layer like I did for 1.0.50)

Both luci-app-statistics and collectd have a new option to select the rrdtool version.
Both options need to be selected if the new rrdtool is used. They default to the old one.

Below are links to my Github repos, where there are rrdtool180 branches in packages and luci:

3 commits in the packages repo
https://github.com/hnyman/packages/commits/rrdtool180

  • Importing the needed graph libraries (fribidi, harfbuzz, cairo, pango)
    (I didn't use the versions in the video feed, as they seemed to be too beefed up for this purpose)
  • rrdtool 1.8.0
  • collectd modifications

1 commit for LuCI:
https://github.com/hnyman/luci/commits/rrdtool180

  • select the correct rrdtool version for luci-app-statistics

The LuCI part is not needed if you are using collectd separately from LuCI stats.

Note that this is just a proof-of-concept, not really a final version.


3 Likes