Collectd-mod-snmp - I dont know where to start

stilgar · October 5, 2018, 10:57am

Hi all,

I Started using luci-statistics and i must say its amazin. Now, im using collectd-mod-rrdtool to get stats from other Openwrt devices inside my main router so i can see how they works without login them.

I have an smart switch i can access snmp to check how its working and i would love to get its stats inside luci-stast as i did with openwrt devices i have. (i cannot install openwrt inside this switch. TL-SG2008)

My main problem is i dont know how to start, since i cannot find any option inside luci ui in order to configure the oid's i want to check.

I searched a lot in internet with no luck. So you are my last chance... do you know where to find a guide or maybe config sample of this mod?

Regards,

lleachii · October 5, 2018, 1:33pm

I'm somewhat lost at the basis of your post.

But for me to collect stats from an OpenWrt router via SNMP, I merely install snmpd.

Hope this helps.

stilgar · October 5, 2018, 4:56pm

Im so sorry for my poor english, please be patient

I understood that collectd-mod-snmp would query stats via snmp from other devices to Openwrt, and create the proper graph inside luci-statistics web... Am i wrong?

I dont want to query openwrt devices, i want my main router query non-openwrt devices via snmp and build proper stats.

Regards,

stilgar · October 6, 2018, 3:39pm

More help, pls!

vgaetera · October 7, 2018, 12:03am

stilgar · October 7, 2018, 10:25am

Yes i found this webpage, but the main problem is that the etc/collectd.conf is a file which must be generated via /etc/config/luci_statistics file once the plugin is loaded.

I tried to imagine how the luci_statistics file should be: (just an example)

config statistics 'collectd_snmp'
option enable '1'

config collectd_snmp_data
       option name 'std_traffic'
       option type 'if_octets'
       option table '1'
       option instance 'IF-MIB::ifDescr'
       option values 'IF-MIB::ifInOctets'

config collectd_snmp_host
       option name 'Host1 InterfaceIn'
       option address 'Host1'
       option version '2'
       option community 'ro_community'
       option collect 'std_traffic'

so i imagined i could get this collect.conf result:

LoadPlugin snmp
<Plugin snmp>
<Data std_traffic>
  type "if_octects"
  table "true"
  instance "IF-MIB::ifDescr"
  values "IF-MIB::ifInOctets"
  </Data>
<Host Host1>
  address "Host1"
  version "2"
  community "ro_community"
  collect "std_traffic"
</Host>
</Plugin>

as i didnt get the info from any guide (just my imagination ), im sure im doing syntax errors i need to solve. The luci_statistics config file seems not to be working since no "snmp" config is translated into collectd.config file, and the plugin isnt loaded. (i typed "collectd" command and this plugins seems to be ignored.

Any clue?

Regards,

vgaetera · October 7, 2018, 10:39am

Include "/etc/collectd/conf.d"

stilgar · October 8, 2018, 7:42am

I dont understand.... /etc/collectd/conf.d" is an empty file, since the config file that governs the collectd behaviour ir /etc/collectd.conf which is automatically generatad due /etc/config/luci_statistics file.

Like I said, i have this file empty, and all other collectd-mod are working correctly.

Could you clarify what you wanted to teach me?

vgaetera · October 8, 2018, 9:25am

In the example above /etc/collectd/conf.d is not a file, but a directory.
The point is create empty directory, containing plugin configuration files with extension .conf in the format of collectd.
Then include that directory in the main configuration with directive Include.

stilgar · October 8, 2018, 9:42am

aahh... So sorry.... i missunderstood. Ill give this a try and let you know if worked or not.

Thanks anyway for the clarification

stilgar · October 8, 2018, 11:16am

Nice, thanks to vgaetera, seems im on the good way.

I created a new file inside /etc/collectd/conf.d/ with .conf extension and, once i solved some sintax problem, seems collectd is doing a work.

I now can find inside /tmp/rrd/Host1/snmp, several files for each interface i queried with snmp.
Now, i can select Host1 inside luci_statistics rrd-tool, to swap from my openwrt device to the one i want to see the graphs.

At this point, i have 2 problems.

1.- When i click "DISPLAY HOST" inside luci graphs web pages, i get a 404 since openwrt device doesnt have "snmp" tab, and the snmp device doesnt have "interfaces" or "cpu" or... If both devices dont share a tab, you will get 404 since, by default, luci tries to load same tab for the new device.
2.- When i correct the path inside chrome nav (typing the tab i want to load: "snmp") so i can get the info, i get no graphs, maybe i need to do something inside luci to get the collected data inside the graphs?

Regards,

stilgar · October 10, 2018, 7:33am

hmm... maybe i did a confusing question... let me clarify.

Im pretty sure im collecting snmp data properly, but i dont know how to check it. Do anybody know which is the correct url to see the snmp graphs?

Regards,

vgaetera · October 10, 2018, 9:20am

# uci get luci_statistics.collectd_rrdtool.DataDir
/tmp/rrd

# uci get luci_statistics.rrdtool.image_path
/tmp/rrdimg

Merge those directories for local and remote host and it should work.

Note, that subdirectories are supposed to be hostnames:

uci get system.@system[0].hostname

stilgar · October 11, 2018, 11:09am

Sadly, it didnt work.

Another clue.... since i merge images and data collectd folder, i noticed that the img file is created when you try to load inside Luci web page. This is the normal behaviour. But this is not happening with snmp. when i try to fix the url to match where the graphs are stored, as i guessed.

If i load "interfaces", suddendly a new img file appears, but not with snmp.

Regards,

jow · October 11, 2018, 11:24am

LuCI needs specific support for each plugin. Without a Lua data model class for snmp, it will not be able to assemble an rrdtool command line to render the image.

vgaetera · October 11, 2018, 11:29am

HOST="$(uci get system.@system[0].hostname)"
RRD="$(uci get luci_statistics.collectd_rrdtool.DataDir)"
RRDIMG="$(uci get luci_statistics.rrdtool.image_path)"
cp -R "$RRD/$HOST" "$RRD/test"
cp -R "$RRDIMG/$HOST" "$RRDIMG/test"

Do this and check some collectd graph in LuCI.
There should be drop-down menu to select a host.

stilgar · October 11, 2018, 1:44pm

I will try to clarify with config files and examples

root@MainHost:~# uci get luci_statistics.collectd_rrdtool.DataDir
/mnt/MyData/System/Collectd
root@MainHost:~# uci get luci_statistics.rrdtool.image_path
/mnt/MyData/System/Collectd

root@MainHost:/mnt/MyData/System/Collectd# ls -lah
drwxrwxrwx    7 root     root        4.0K Oct 11 15:25 .
drwxr-xr-x    4 root     root        4.0K Feb 27  2018 ..
drwxrwxrwx   23 root     root        4.0K Oct 10 12:16 RRDHost1
drwxrwxrwx   32 root     root        4.0K Oct 10 12:13 MainHost
drwxrwxrwx    3 root     root        4.0K Oct 10 12:12 SNMPHost
drwxr-xr-x   32 root     root        4.0K Oct 11 15:25 Test
drwxrwxrwx   12 root     root        4.0K Oct 10 12:12 RRDHost2

MainHost = Is the router where im going to collect all data, rrd and snmp.
RRDHost1 & RRDHost2 = Hosts with openwrt that are sending its stats to MainHost.
Test = The Folder i created doing what you asked for last post.
SNMPHost = The collectd-mod-snmp created this folder with the snmp queries.

root@MainHost:/mnt/MyData/System/Collectd/SNMPHost/snmp# ls -lah
drwxrwxrwx    2 root     root        4.0K Oct 10 12:12 .
drwxrwxrwx    3 root     root        4.0K Oct 10 12:12 ..
-rwxrwxrwx    1 root     root       18.7K Oct 11 15:30 if_octets-AUX.rrd
-rwxrwxrwx    1 root     root       18.7K Oct 11 15:30 if_octets-Loopback.rrd
-rwxrwxrwx    1 root     root       18.7K Oct 11 15:30 if_octets-Vlan-interface1.rrd
-rwxrwxrwx    1 root     root       18.7K Oct 11 15:30 if_octets-port 1: Gigabit Copper.rrd
-rwxrwxrwx    1 root     root       18.7K Oct 11 15:30 if_octets-port 2: Gigabit Copper.rrd
-rwxrwxrwx    1 root     root       18.7K Oct 11 15:30 if_octets-port 3: Gigabit Copper.rrd
-rwxrwxrwx    1 root     root       18.7K Oct 11 15:30 if_octets-port 4: Gigabit Copper.rrd
-rwxrwxrwx    1 root     root       18.7K Oct 11 15:30 if_octets-port 5: Gigabit Copper.rrd
-rwxrwxrwx    1 root     root       18.7K Oct 11 15:30 if_octets-port 6: Gigabit Copper.rrd
-rwxrwxrwx    1 root     root       18.7K Oct 11 15:30 if_octets-port 7: Gigabit Copper.rrd
-rwxrwxrwx    1 root     root       18.7K Oct 11 15:30 if_octets-port 8: Gigabit Copper.rrd

This is the config for collectd-snmp: (inside /etc/collectd/conf.d/)

LoadPlugin snmp
<Plugin snmp>
<Data std_traffic>
 Type "if_octets"
 Table true
 Instance "IF-MIB::ifName"
 Values "IF-MIB::ifInOctets" "IF-MIB::ifOutOctets"   
</Data>
<Host "SNMPHost">
 address "SNMPHost"
 version 2
 community "ROComm"
 collect "std_traffic"
</Host>
</Plugin>

After i tried to use snmp plugin, i already got the dropdown menu with the stats, since other Openwrt Devices were sending its stats to MainHost. (with rrdtools)

This is the url i got when i read stats inside MainHost:
http://MainHost/cgi-bin/luci/admin/statistics/graph/cpu

This is the url i got when i read stats inside MainHost, but using the dropdown menu RRDHost1:
http://MainHost/cgi-bin/luci/admin/statistics/graph/cpu?host=RRDHost1&submit=Display+Host+»&timespan=1hour

When i click "DISPLAY HOST" for the SNMPHost, i got a 404, since it tries:
http://MainHost/cgi-bin/luci/admin/statistics/graph/cpu?host=SNMPHost&submit=Display+Host+»&timespan=1hour

ITs logic, since SNMPHost isnt getting queries for "cpu" yet (exaclty as when i ask for items that were not quering inside RRDHostx), so i tried chaning to snmp this way:
http://MainHost/cgi-bin/luci/admin/statistics/graph/snmp?host=SNMPHost&submit=Display+Host+»&timespan=1hour

Then i got no 404, but got no graph neither... Maybe the url should be??

Regards,

stilgar · October 15, 2018, 1:32pm

Pls, any idea?

vgaetera · October 15, 2018, 2:13pm

You need to compare the structure of directories MainHost and SNMPHost.
Compare file names and data containing inside those files.
It might help to find a way to feed the data to luci-statistics.

Sorry, the problem is I do not use collectd, just installed it out of curiosity.

stilgar · October 17, 2018, 9:08am

I finally got the solution. I posted into luci github forum, and they told me that "luci-app-statistics" doesnt support collectd-mod-snmp. And they invited me to see the data model of any other collectd-mod-xxx in order to create my own collectd-mod-snmp data model so luci-app-statistics should support it.

Since snmp is so wide, its impossible to get a closed data model, but i think i got a workarround.

firstly, its quite important install collectd-mod-syslog in order to read all log entries so you can see which modules are needed in order to get the graph working.

Later, you can use all collectd-mod-xxx which are already supported, in order to simulate same folder structure. For example, i queried via snmp the uptime of the device. As i already had installed collectd-mod-uptime, there was really easy make it works.

this is the folder content of a collectd-mod-uptime device:

$Collectd_Folder/rrd_device/uptime/uptime.rrd

This is the folder content of the snmp device:

$Collectd_Folder/snmp_device/smmp/uptime.rrd

The final solution is to create a symlink this way:

$Collectd_Folder/snmp_device/smmp/uptime.rrd
$Collectd_Folder/snmp_device/uptime/uptime.rrd -> $Collectd_Folder/snmp_device/smmp/uptime.rrd

Now, luci-app-statistics, draw the graph as this snmp device, really was created with the collectd-mod-uptime instead.

So, be carefull to query via snmp same values that collectd-mod-xxxxxx provides to luci-app-statistics. (thats why you must install collectd-mod-syslog, so you can know which files collectd-mod-xxxx are expecting, or you can also see the proper datamodel to see which files are generated)

Regards,