Exporting Openwrt interface and performance statistics to Influxdb and Grafana - instruction

Community Builds, Projects & Packages
1

I created instruction how to export Openwrt statistics (collected by Collectd) to Influxdb and then display them in Grafana. The graphs are almost realtime - the level of detail is 1 minute for last 24 hours. The statistics are downsampled what allows you to view information what was the system performance in the past (several month or years ago) while still keeping database size small. Of course level of details for statistics from one year ago is much lower (6 hours) that for the statistics for last 24 hours (1 minute).

The instruction can be found at github

Example statistics:

Data is aggeregated and downsampled over time. Following aggregations are set:

Daily - data resolution is 1 minute
Weekly - data resolution is 5 minutes
Monthly - data resolution is 30 minutes
Yearly - data resolution is 6 hours
10 Year - data resolution is 1 day
100 Year - data resolution is 1 week 
Original data send from collectd is store in Collect/autogen bucket
5 Likes
2

Interesting job, thanks. Maybe I would like to try it. I need:

  1. a list on the collectd enabled features in the openwrt menuconfig. or share the openwrt config file(.confg).
  2. a list on all special IP ports used by every software.
  3. share a ready-to-use RaspberryPI SDCard image if possible, thanks.
3

Collectd enabled features

root@Openwrt:~# cat /etc/collectd.conf
BaseDir "/var/run/collectd"
Include "/etc/collectd/conf.d/*.conf"
PIDFile "/var/run/collectd.pid"
PluginDir "/usr/lib/collectd"
TypesDB "/usr/share/collectd/types.db"
Interval 30
ReadThreads 2


LoadPlugin iptables

LoadPlugin conntrack

LoadPlugin iwinfo

LoadPlugin memory
<Plugin memory>
        ValuesPercentage false
        ValuesAbsolute true
</Plugin>

LoadPlugin cpu
<Plugin cpu>
        ValuesPercentage true
        ReportByCpu true
        ReportByState true
</Plugin>

LoadPlugin load

LoadPlugin rrdtool
<Plugin rrdtool>
        DataDir "/tmp/rrd"
        RRARows 288
        RRASingle true
        RRATimespan 3600
        RRATimespan 86400
        RRATimespan 604800
        RRATimespan 2678400
        RRATimespan 31622400
</Plugin>

LoadPlugin uptime

LoadPlugin interface
<Plugin interface>
        Interface "br-lan"
        Interface eth0
        Interface eth1
        Interface lan1
        Interface lan2
        Interface lan3
        Interface lan4
        Interface wan
        Interface wlan0
        Interface wlan1
</Plugin>

LoadPlugin dns
<Plugin dns>
        Interface "br-lan"
        IgnoreSource "127.0.0.1"
</Plugin>

LoadPlugin network
<Plugin network>
        Server "192.168.100.99" "25827"
</Plugin>

LoadPlugin dhcpleases
<Plugin dhcpleases>
        Path "/tmp/dhcp.leases"
</Plugin>

root@Openwrt:~#

Used ports:

  • Grafana: 3000
  • Collectd: 25826
  • Telegraf: 25827
  • Influxdb: 8086

Raspberrypi image

I do not have any image to share. However you can try to use any system and try to configure Docker. For the InfluxDB and Telegraf I used following commands:

  • Get collectd types.db
wget https://raw.githubusercontent.com/collectd/collectd/main/src/types.db
  • Configure Telegraf
cat  $PWD/telegraf/telegraf.conf
[[inputs.socket_listener]]
service_address = "udp://:25826"
data_format = "collectd"
collectd_typesdb = ["/usr/share/collectd/types.db"]
collectd_parse_multivalue = "split"

[inputs.socket_listener.tags]
  bucket = "collectd"

[[outputs.influxdb_v2]]
urls = ["http://influxdb:8086"]
token = "$DOCKER_INFLUXDB_INIT_ADMIN_TOKEN"
organization = "$DOCKER_INFLUXDB_INIT_ORG"
bucket = "collectd/autogen"

[outputs.influxdb_v2.tagpass]
  bucket = ["collectd"]
  • Create internal network
docker network create --driver bridge influxdb-telegraf-net
  • Create influxdb container
docker run -d -p 8086:8086 \
  --name=influxdb \
  --net=influxdb-telegraf-net \
  -v "$PWD/data:/var/lib/influxdb2" \
  -v "$PWD/config:/etc/influxdb2" \
  -e DOCKER_INFLUXDB_INIT_MODE=setup \
  -e DOCKER_INFLUXDB_INIT_USERNAME=admin \
  -e DOCKER_INFLUXDB_INIT_PASSWORD=password \
  -e DOCKER_INFLUXDB_INIT_ORG=my-org \
  -e DOCKER_INFLUXDB_INIT_BUCKET=my-bucket \
  -e DOCKER_INFLUXDB_INIT_RETENTION=1w \
  -e DOCKER_INFLUXDB_INIT_ADMIN_TOKEN=my-admin-token \
  influxdb:2
  • Create Telegraf container:
docker run -d --name=telegraf \
      -p 25826:25826 \
      -v "$PWD/telegraf/telegraf.conf:/etc/telegraf/telegraf.d/telegraf.conf" \
      --net=influxdb-telegraf-net \
	  -v "$PWD/types.db:/usr/share/collectd/types.db" \
	  -e DOCKER_INFLUXDB_INIT_ORG=my-org \
	  -e DOCKER_INFLUXDB_INIT_ADMIN_TOKEN=my-admin-token \
      telegraf
  • Create Grafana container
Unfortunatelly I do not have command here, you need to Google this

After executing all those steps you will have working Grafana/Influxd/Telegraf. This will allow you to proceed with my instruction.

1 Like
4

Hi, @mstojek , I've try to install a system on Raspberry Pi3. I got all software running, but cannot receive the data from openwrt.
It seems no data into the influxdb, could you help?

containers

Screenshot from 2024-08-24 16-46-31
Screenshot from 2024-08-24 16-46-311233×709 173 KB

telegraf logs

2024-08-24T06:28:25Z I! Loading config: /etc/telegraf/telegraf.conf
2024-08-24T06:28:25Z I! Starting Telegraf 1.31.3 brought to you by InfluxData the makers of InfluxDB
2024-08-24T06:28:25Z I! Available plugins: 234 inputs, 9 aggregators, 32 processors, 26 parsers, 60 outputs, 6 secret-stores
2024-08-24T06:28:25Z I! Loaded inputs: socket_listener
2024-08-24T06:28:25Z I! Loaded aggregators:
2024-08-24T06:28:25Z I! Loaded processors:
2024-08-24T06:28:25Z I! Loaded secretstores:
2024-08-24T06:28:25Z I! Loaded outputs: influxdb_v2
2024-08-24T06:28:25Z I! Tags enabled: host=af798d79cc31
2024-08-24T06:28:25Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"af798d79cc31", Flush Interval:10s
2024-08-24T06:28:25Z I! [inputs.socket_listener] Listening on udp://[::]:25827
2024-08-24T06:46:09Z I! [agent] Hang on, flushing any cached metrics before shutdown
2024-08-24T06:46:09Z I! [agent] Stopping running outputs
2024-08-24T06:47:10Z I! Loading config: /etc/telegraf/telegraf.conf
2024-08-24T06:47:10Z I! Starting Telegraf 1.31.3 brought to you by InfluxData the makers of InfluxDB
2024-08-24T06:47:10Z I! Available plugins: 234 inputs, 9 aggregators, 32 processors, 26 parsers, 60 outputs, 6 secret-stores
2024-08-24T06:47:10Z I! Loaded inputs: socket_listener
2024-08-24T06:47:10Z I! Loaded aggregators:
2024-08-24T06:47:10Z I! Loaded processors:
2024-08-24T06:47:10Z I! Loaded secretstores:
2024-08-24T06:47:10Z I! Loaded outputs: influxdb_v2
2024-08-24T06:47:10Z I! Tags enabled: host=af798d79cc31
2024-08-24T06:47:10Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"af798d79cc31", Flush Interval:10s
2024-08-24T06:47:10Z I! [inputs.socket_listener] Listening on udp://[::]:25827

collectd config

Screenshot from 2024-08-24 15-48-39
Screenshot from 2024-08-24 15-48-391069×333 32.7 KB

5

For sure lack of udp in telegraf container is an issue.
Unfortunatelly I never fully set up this with Docker so I can not help you to solve docker issues here. On regular linux machine I would use tcpdump to check if collect is sending anything to UDP telegraf port, but with docker it might more tricky to debug this.

6

Thanks, I'll try to create a new telegraf container with:

docker run -d --name=telegraf \
      -p 25827:25827/udp \
.............
7

Finally, it all worked OK:

Summary

Screenshot from 2024-08-24 21-52-26
Screenshot from 2024-08-24 21-52-261839×814 229 KB

Thanks.

1 Like
8

Good to hear that you were able to set up everything in Docker!

9

Thanks for this inspiration!

I noticed that collectd has some additional plugins on openwrt and some others are missing.

Why did you use telegraf before influxdb? Can't collectd write directly into influxdb?

I'm currently working on something similar, because influxdb made some weird changes to their open-source policies with version 3. Instead, I tried victoriametrics. And then I found there is collectd6 that uses standardized metrics naming based on the open telemetry standard.

10

While I used prometheus instead, during my research influxdb came up, and I remember reading that as of 2.x there is no native collectd support in influxdb, thus the need for telegraf or another "collector" in front of the database.

11

maybe nodered ->influxdb->grafana
why not ????

12

There is Also a variant