Here a partial update to luci statistics for mqtt

hi there here a partial update for luci statistic to include mqtt - though i am not sure in collectd Mqtt is working for me as i have not created a graph yet or create a mqtt rrd

  1. edit /usr/bin/stat-genconfig

     function config_mqtt( c )
     local str = ""
    
    
     for s in pairs(sections) do
    
    
    
         for key, type in pairs({ Sub="collectd_mqtt_sub", Pub="collectd_mqtt_pub" }) do
                 if sections[s][".type"] == type then
    
                         sname = sections[s].sname
                         host = sections[s].host
                         user = sections[s].user
                         port = sections[s].port
                         topic = sections[s].topic
                         password = sections[s].password
                         pname = sections[s].pname
                         prefix = sections[s].prefix
    
    
                         if sname then
                                 if user then
                                         str = str .. "\t" .. "<Subscribe " .."\"".. sname .."\"".." >" .. " \n"
    
                                         str = str .. "\t" .. "host" .. " \"" .. host .. "\"\n"
                                         str = str .. "\t" .. "port" .. " \"" .. port .. "\"\n"
                                         str = str .. "\t" .. "user" .. " " .. user .. "\n"
                                         str = str .. "\t" .. "password" .. " " .. password .. "\n"
                                         str = str .. "\t" .. "topic" .. " \"" .. topic .. "\"\n"
                                          str = str .. "\t" .. "CleanSession true" .. " \n"
                                          str = str .. "\t" .. "</Subscribe>" .. " \n"
    
    
                                 else
                                         str = str .. "\t" .. "<Subscribe " .."\"".. sname .."\"".." >" .. " \n"
    
                                         str = str .. "\t" .. "host" .. " \"" .. host .. "\"\n"
                                         str = str .. "\t" .. "port" .. " \"" .. port .. "\"\n"
                                         str = str .. "\t" .. "topic" .. " \"" .. topic .. "\"\n"
                                         str = str .. "\t" .. "</Subscribe>" .. " \n"
    
                                 end
    
    
                                    else
                                         str = str .. "\t" .. "<Publish " .."\"".. pname .."\"".." >" .. " \n"
    
                                         str = str .. "\t" .. "host" .. " \"" .. host .. "\"\n"
                                         str = str .. "\t" .. "port" .. " \"" .. port .. "\"\n"
                                        str = str .. "\t" .. "prefix" .. " \"" .. prefix .. "\"\n"
                                         str = str .. "\t" .. "</Publish>" .. " \n"
    
                                 end
                      
                     end
    
                         end
         end
     return str
    
    end
    

and add this to plugin section

  mqtt = config_mqtt,

created /usr/lib/lua/luci/model/cbi/luci_statistics/mqtt.lua

-- Copyright 2008 Freifunk Leipzig / Jo-Philipp Wich <jow@openwrt.org>
-- Licensed to the public under the Apache License 2.0.

m = Map("luci_statistics",
        translate("MQTT Subscriptions"),
        translate(
                "The MQTT will subscribe to MQTT Topics " ..
                "to be displayled "
        ))



-- collectd_mqtt config section
s = m:section( NamedSection, "collectd_mqtt", "luci_statistics" )

-- collectd_mqtt_sub.enable
enable = s:option( Flag, "enable", translate("Enable this plugin") )
enable.default = 0


-- collectd_mqtt_sub config section (Sub)
sub = m:section( TypedSection, "collectd_mqtt_sub",
        translate("MQTT Subscriptions"),
        translate(
                "This section defines on which subscription collectd will wait " ..
                "for incoming data."
        ))
sub.addremove = true
sub.anonymous = true


-- collectd_mqtt_sub.sname
sub_name = sub:option( Value, "sname", translate("Subscription Name"))
sub_name.default = "Subscription1"


-- collectd_mqtt_sub.hosts
sub_hosts = sub:option( Value, "host", translate("MQTT hosts"))
sub_hosts.default = "127.0.0.1"


-- collectd_mqtt_sub.port
sub_port = sub:option( Value, "port", translate("MQTT port") )
sub_port.isinteger = true
sub_port.default   = "1883"


-- collectd_mqtt_sub.user
sub_user = sub:option( Value, "user", translate("MQTT User") )
sub_user.isinteger = true
sub_user.default   = ""


-- collectd_mqtt_sub.password
sub_password = sub:option( Value, "password", translate("MQTT password") )
sub_password.isinteger = true
sub_password.default   = ""




-- collectd_mqtt_sub.topic
sub_topic = sub:option( Value, "topic", translate("Topic to Subscribe too") )
sub_topic.isinteger = true
sub_topic.default   = "incoming/#"


-- collectd_mqtt_pub config section (Pub)
pub = m:section( TypedSection, "collectd_mqtt_pub",
        translate("Publish MQTT"),
        translate(
                "This section defines to which MQTT servers it will publish MQTT " ..
                "data to."
        ))
pub.addremove = true
pub.anonymous = true

-- collectd_mqtt_pub.pname
pub_name = pub:option( Value, "pname", translate("Publish Name"))
pub_name.default = "PublishCollectd"


-- collectd_mqtt_pub.host
pub_host = pub:option( Value, "host", translate("MQTT  host") )
pub_host.default = "localhost"

-- collectd_mqtt_pub.port
pub_port = pub:option( Value, "port", translate("MQTT  port") )
pub_port.default   = ""
pub_port.isinteger = true
pub_port.optional  = true

-- collectd_mqtt_pub.prefix
pub_prefix = pub:option( Value, "prefix", translate("Prefix") )
pub_prefix.default = "collectd"


return m

added the /usr/lib/lua/luci/controller/luci_statistics/luci_statistics.lua

     mqtt            = _("MQTT"),

and added mqtt to this section

   network = { "conntrack", "dns", "interface", "iptables", "mqtt",
                    "netlink", "olsrd", "openvpn", "ping",
                    "splash_leases", "tcpconns", "iwinfo" }

edit /etc/config/luci_statistics

  config statistics 'collectd_mqtt'
   option enable '1'

It creates the proper config and collectd starts with out errors

I am uncertain how to create a graph this is where I am mainly stuck

create /usr/lib/lua/luci/statistics/rrdtool/definitions/mqtt.lua

also I am uncertain if mqtt plugin is working does it automatically produce a rrd container or it has to be explicitly told to create a rrd container in openwrt ????

okay i can verify the mqtt is working for publish via my update for luci statistics--

Screenshot_20180308_134457

example output:

root@LEDE:/etc# mosquitto_sub -t 'collectd/#' -v
collectd/LEDE/iwinfo-wlan0/bitrate 1520537314.863:0
collectd/LEDE/iwinfo-wlan0/signal_power 1520537314.863:0
collectd/LEDE/cpu-0/cpu-user 1520537314.866:0.133352650565559
collectd/LEDE/iwinfo-wlan0/signal_quality 1520537314.864:0
collectd/LEDE/cpu-1/cpu-user 1520537314.866:0.266705165168375
collectd/LEDE/cpu-0/cpu-system 1520537314.866:0.700101095498378
collectd/LEDE/cpu-1/cpu-system 1520537314.866:0.200028905823664
collectd/LEDE/cpu-0/cpu-wait 1520537314.866:0
collectd/LEDE/cpu-1/cpu-wait 1520537314.866:0.0333381564359501
collectd/LEDE/cpu-0/cpu-nice 1520537314.866:0
collectd/LEDE/cpu-1/cpu-nice 1520537314.866:0
collectd/LEDE/cpu-0/cpu-interrupt 1520537314.866:0
collectd/LEDE/cpu-1/cpu-interrupt 1520537314.866:0
collectd/LEDE/cpu-0/cpu-softirq 1520537314.866:1.76692239912554
collectd/LEDE/cpu-1/cpu-softirq 1520537314.866:0.633425045306426
collectd/LEDE/cpu-0/cpu-steal 1520537314.866:0
collectd/LEDE/cpu-1/cpu-steal 1520537314.866:0
collectd/LEDE/iwinfo-wlan0/signal_noise 1520537314.864:0
collectd/LEDE/cpu-0/cpu-idle 1520537314.866:90.6455411358221
collectd/LEDE/cpu-1/cpu-idle 1520537314.866:95.4128436188252
collectd/LEDE/iwinfo-wlan0/stations 1520537314.874:0
collectd/LEDE/interface-br-lan/if_octets 1520537314.874:40551.4731501939:2256778.90849703
collectd/LEDE/interface-br-lan/if_errors 1520537314.874:0:0
collectd/LEDE/interface-br-lan/if_dropped 1520537314.875:0:0
collectd/LEDE/interface-br-lan/if_packets 1520537314.874:809.573134264416:1500.59329864087
collectd/LEDE/iwinfo-wlan1/bitrate 1520537314.882:0
collectd/LEDE/iwinfo-wlan1/signal_power 1520537314.888:0
collectd/LEDE/iwinfo-wlan1/signal_noise 1520537314.889:0
collectd/LEDE/iwinfo-wlan1/signal_quality 1520537314.890:0
collectd/LEDE/iwinfo-wlan1/stations 1520537314.890:0
collectd/LEDE/iwinfo-wlan0/bitrate 1520537344.760:0
collectd/LEDE/iwinfo-wlan0/stations 1520537344.761:0
collectd/LEDE/iwinfo-wlan0/signal_noise 1520537344.761:0
collectd/LEDE/iwinfo-wlan0/signal_quality 1520537344.761:0
collectd/LEDE/iwinfo-wlan0/signal_power 1520537344.760:0
collectd/LEDE/iwinfo-wlan1/bitrate 1520537344.770:0
collectd/LEDE/iwinfo-wlan1/signal_power 1520537344.770:0
collectd/LEDE/iwinfo-wlan1/signal_noise 1520537344.770:0
collectd/LEDE/iwinfo-wlan1/signal_quality 1520537344.773:0
collectd/LEDE/iwinfo-wlan1/stations 1520537344.774:0

which is great i can now display this info on TFT screens connected by esp8266 screen.. that provides live network info using lede and collectd as the provider of info see - https://github.com/krywenko/ESP_TFT_GRAPH_MQTT for details how

but subscribing is still being elusive -- if the publish is working subscribing should too i would think..

would any one be willing to provide a little insight to where the problem lies

I am passing the default example from collectd wiki :

mosquitto_pub -t 'incoming/localhost/mqtt/temperature-kitchen' -m 'N:21.5'
and just incase it wanted host specific 
mosquitto_pub -t 'incoming/LEDE/mqtt/temperature-kitchen' -m 'N:21.5'

but I do not see any RRD containers being produced in the RRD folder,, would anyone have a suggestion where to look or what might be wrong ??????

would it be related to this /usr/lib/lua/luci/statistics/rrdtool/definitions/mqtt.lua as I am not sure if a created a proper one but I thought that was more for displaying in luci then creating rrd but perhaps I am wrong

for those who might be interested I downloaded the newest version of openwrt where collectd-mod-mqtt is part of software packages followed the above and now it produces MQTT rrd container that outputs to the CSV via the statistic plugin - just need to work on the graph part and it should work fine.
sample output from colectd-csv- output
mqtt

Which is...? snapshot or 18.06.0?

just the stable 18.06.0

i got it graph now too, I would copy the mqtt.lua here but for some reason I can not get it to mount a usb did that change as it now says invalid argument since the last version

mqtts

okay figured out why i could not mount it here a copy of the mqtt.lua
just copy it to here
nano /usr/lib/lua/luci/statistics/rrdtool/definitions/mqtt.lua
it only seams to work if you use ssl mosquitto



module("luci.statistics.rrdtool.definitions.mqtt", package.seeall)

function rrdargs(graph, plugin, plugin_instance)
    -- For $HOSTNAME/exec-foo-bar/temperature_baz-quux.rrd, plugin will be
    -- "exec" and plugin_instance will be "foo-bar".  I guess graph will be
    -- "baz-quux"?  We may also be ignoring a fourth argument, dtype.
if "Energy" == plugin_instance then

        --
        -- Energy diagram
        --
 local energy = {

                -- draw this diagram for each plugin instance
               per_instance = true,    -- enable or disable for different graphing
                title = "%H: Energy Production and Consumption--%di",
                vlabel = "watts",
number_format = "%6.2lf%%",
                -- diagram data description
                data = {
                        -- defined sources for data types, if ommitted assume a single DS named "value" (optional)
                        sources = {
                                power = { "value"}
                        },

                        -- special options for single data lines
                        options = {

                                --  power_wind_value = {
                                --  title = "wind power",
                                --  color = "00e000" -- green
                               
                                                               
                               power__value = {
                                              total = true,           -- report total amount
                                            --   color = "0000ff",       -- rx is blue
                                            --   noarea = true,
                                                overlay = true,
                                                title = "%di"
--transform_rpn = "3600,/"
                               }
                        }
                }
        }
 return { energy }
end

if "Temp" == plugin_instance then
 --
        -- Temperature diagram
        --
        local temp = {

                -- draw this diagram for each plugin instance
          --      per_instance = true,     -- enable or disable for different graphing
                title = "%H: Temperature %di",
                vlabel = "celsius",

                -- diagram data description
                data = {
                        -- defined sources for data types, if ommitted assume a single DS named "value" (optional)
                        sources = {
                                temperature = { "value" }
                        },

                        -- special options for single data lines
                        options = {
                                temperature__value = {
                                --        total = true,           -- report total amount
                               --         color = "0000ff",       -- rx is blue
                                       noarea = true,
                                       overlay = true,
                                       title = "%di"
                                
                                }
                        }
                }
        }
 return { temp }

end
                                
                                

if "Flow" == plugin_instance then
 --
        -- Flow diagram
        --
        local flow = {

                -- draw this diagram for each plugin instance
                per_instance = true,     -- enable or disable for different graphing
                title = "%H: Flow - %di",
                vlabel = "litre",

                -- diagram data description
                data = {
                        -- defined sources for data types, if ommitted assume a single DS named "value" (optional)
                        sources = {
                                flow = { "value" }
                        },

                        -- special options for single data lines
                        options = {
                                flow__value = {
                                --        total = true,           -- report total amount
                                        color = "ff00ff",
                                        area =  "00ff00",       -- rx is blue
                                      -- noarea = true,
                                       overlay = true,
                                       title = "%di"
                               
                                }
                        }
                }
        }

 return { flow }

end

if "Humidity" == plugin_instance then
 --
        -- Temperature diagram
        --
        local humid = {

                -- draw this diagram for each plugin instance
          --      per_instance = true,     -- enable or disable for different graphing
                title = "%H: Humidity - %di",
                vlabel = "percent",

                -- diagram data description
                data = {
                        -- defined sources for data types, if ommitted assume a single DS named "value" (optional)
                        sources = {
                                humidity = { "value" }
                        },

                        -- special options for single data lines
                        options = {
                                humidity__value = {
                                --        total = true,           -- report total amount
                                        color = "1100ff",       -- rx is blue
                                      -- noarea = true,
                                       overlay = true,
                                       title = "%di"
                                 --    title = "Humidity"
                                }
                        }
                }
        }
 return { humid }

end

if "Pressure" == plugin_instance then
 --
        -- Temperature diagram
        --
        local press = {

                -- draw this diagram for each plugin instance
          --      per_instance = true,       -- enable or disable for different graphing
                title = "%H: Pressure - %di",
                vlabel = "kPa",

                -- diagram data description
                data = {
                        -- defined sources for data types, if ommitted assume a single DS named "value" (optional)
                        sources = {
                                pressure = { "value" }
                        },

                        -- special options for single data lines
                        options = {
                                pressure__value = {
                                --        total = true,           -- report total amount
                               --         color = "0000ff",       -- rx is blue
                                       noarea = true,
                                       overlay = true,
                                       title = "%di"
                               
                                }
                        }
                }
        }
 return { press }

end


end

format for sending topic
mosquitto_pub -t 'incoming/OpenWrt/mqtt-Energy/power-grid' -m 'N:21.5'
mosquitto_pub -t 'incoming/OpenWrt/mqtt-Temp/temperature-greenhouse' -m 'N:21.5'
mosquitto_pub -t 'incoming/OpenWrt/mqtt-Humidity/humidity-greenhouse' -m 'N:21.5'
mosquitto_pub -t 'incoming/OpenWrt/mqtt-Flow/flow-heatpump' -m 'N:21.5'
mosquitto_pub -t 'incoming/OpenWrt/mqtt-Pressure/pressure-heatpump' -m 'N:21.5'

i was wondering if the the above will be added to the collectd-mod-mqtt .. to make it easier for others at some point

I tried the code you posted but the tab with MQTT configuration never shows up. I have to do something specific after I modify the files?

Thanks.

I think you probably just missed something or did something incorrect chances are you accidentally added the : at the end of the file name ( which I removed from the how to now as not to confuse others again)

  • nano /usr/bin/stat-genconfig ( this processes the page )
  • nano /usr/lib/lua/luci/model/cbi/luci_statistics/mqtt.lua ( this creates the page )
  • nano /usr/lib/lua/luci/controller/luci_statistics/luci_statistics.lua ( the tell what section the plug is listed and where)
  • nano /etc/config/luci_statistics ( this enables the plugin to be shown)

Hi sash99,

I am using your MQTT statictics with my own setup but something is missing.

I am able to setup the service, so everythings is OK in the SETUP menu. I can configure MQTT and subscribe to topics.

In fact, collectd is collecting all the info properly in RRD files, which have correct data.

But I dont have the MQTT tab in the "Graphs" section.

Here it is the path of one of the RRD files as an example:

OPENWRT/mqtt-cuarto_ninos/temperature-setpoint_CTL.rrd

In the file /usr/lib/lua/luci/statistics/rrdtool/definitions/mqtt.lua:

if "cuarto_ninos" == plugin_instance then

    -- Temperature diagram
    --
    local temp = {

            -- draw this diagram for each plugin instance
      --      per_instance = true,     -- enable or disable for different graphing
            title = "%H: Temperature %di",
            vlabel = "celsius",

            -- diagram data description
            data = {
                    -- defined sources for data types, if ommitted assume a single DS named "value" (optional)
                    sources = {
                            temperature = { "value" }

.....

What could I miss from your setup?

Thank you!

if you use my above mqtt definition does it work for you it should draw a temperature graph for you

1 Like

I like this MQTT UI. It is possible to make it as a luci-app-*.ipk to install it directly using opkg.

if I knew how I would. but I guess you have to wait for maybe openwrt to include it .. but I think it is not likely so you have to manually insert yourself . perhaps if you others also requested it be included maybe they would at some point

Ok, I'm sorry that I don't quite understand the above. Could you please provide a complete process of how to realize this UI again? I tried to modify it for several times, but my Web page could not be accessed. Can you provide your complete documents? Thank you for your help.

The file/usr/lib/lua/luci/model/cbi/luci_statistics MQTT lua
The file /usr/bin/stat-genconfig
The file/usr/lib/lua/luci/controller/luci_statistics/luci_statistics lua
This file is /etc/config/luci_statistics
Some of these files exist on the OP system, others I need to create?
And I don't understand "MQTT = config_MQTT," "MQTT = config_MQTT," "network = {"conntrack"," DNS ", "interface", "iptables", "MQTT ",
Netlink ""," olsrd openvpn ", ""," ping ",

"Splash_leases", "tcpconns iwinfo", ""}"
Exactly where they are added. I need more specific documents. I'm sorry to make such an unreasonable request, but I really need your help.

if I said edit or add in the description above then the file exist, if I say create then the file does not exist and you need to create it
this was based on openwrt 18 it may of changed in later version of openwrt but I never investigated as I am still using openwrt 18.XX as I never had issue and it works brilliantly so i never updated
example :
edit /usr/bin/stat-genconfig = nano /usr/bin/stat-genconfig
in this case I put this after iptables section in this file

in the same file edit plugin section

  mqtt = config_mqtt,

Screenshot_20220330_072358
( in this case you can also see where I added snmp and modbus as well you can ignore that )

created /usr/lib/lua/luci/model/cbi/luci_statistics/mqtt.lua
so nano /usr/lib/lua/luci/model/cbi/luci_statistics/mqtt.lua
this will create the file when you save it.

added the /usr/lib/lua/luci/controller/luci_statistics/luci_statistics.lua
so again


and in the same file
Screenshot_20220330_073105

and then edit /etc/config/luci_statistics = nano /etc/config/luci_statistics
Screenshot_20220330_073713

then you need to create your rdd deffinition in my case I used
nano /usr/lib/lua/luci/statistics/rrdtool/definitions/mqtt.lua

and the format for collectd-mqtt for capturing data is
mosquitto_pub -t 'incoming/OpenWrt/mqtt-Energy/power-grid' -m 'N:21.5'
mosquitto_pub -t 'incoming/OpenWrt/mqtt-Temp/temperature-greenhouse' -m 'N:21.5'
mosquitto_pub -t 'incoming/OpenWrt/mqtt-Humidity/humidity-greenhouse' -m 'N:21.5'
mosquitto_pub -t 'incoming/OpenWrt/mqtt-Flow/flow-heatpump' -m 'N:21.5'
mosquitto_pub -t 'incoming/OpenWrt/mqtt-Pressure/pressure-heatpump' -m 'N:21.5'

i just remember that from another one of my luci apps on a newer version openwrt and it was broken --

and I still never upgraded from 18.xx so I am not sure if it still works in that case just use a older version of openwrt18

Thank you for your help. I will try again and hope it can be completed. Thanks again.