[testers needed] [21.02, snapshot] collectd-exec scripts to collect Lantiq DSL values

What is? I don't see anything I wouldn't have done myself. Of course this only creates numerical rtx values if the JSON value is available, so for those who don't have the current patches it would be sensible to return zero instead of an empty string, i.e.

echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/errors-near_rtx_uc\" N:${errors_near_rtx_uc:-0}"

or something to that effect. But that would be for a "release version" of this script, if there is such a thing.

Edit: This might not even be necessary. If I read the jshn.sh source correctly, json_get_var has an undocumented third parameter for a fallback value. So, untested, for example

json_get_var errors_far_rtx_uc    rx_corrupted 0

should, if it can't find the value, set the variable to 0. No further conditional processing necessary.

Oh, currently the assembled plot is 'ugly', mostly because some colors lines are invisible as they are plotted behind other area plots... I guess I want tobplay a bit with the plotting options....
Thanks for the pointer about how to deal with empty values, will incorporate and post, after some testing.

So here is a version that actually seems to work:
cat /usr/lib/collectd/exec/exec-lantiqdsl.sh

#!/bin/sh

# source jshn shell library
. /usr/share/libubox/jshn.sh

HOSTNAME="${COLLECTD_HOSTNAME:-$(cat /proc/sys/kernel/hostname)}"

# retrieve DSL metrics through ubus
dsl_metrics_json=$(/bin/ubus call dsl metrics)

# initialize JSHN and load JSON
json_init
json_load "$dsl_metrics_json"

# get line state
json_get_var linestate up

# only continue if line is up
# collecting any line stats is pointless if the line is down, downtime will be reflected by the gap in statistics
[ "$linestate" = "1" ] || exit 0

# get basic line stats
json_get_var uptime uptime

# get downstream and upstream stats
json_select downstream
	json_get_var downstream_datarate     data_rate
	json_get_var downstream_datarate_max attndr
	json_get_var downstream_snr          snr
json_close_object
json_select upstream
	json_get_var upstream_datarate       data_rate
	json_get_var upstream_datarate_max   attndr
	json_get_var upstream_snr            snr
json_close_object

# get near and far errors
json_select errors
	json_select near
		json_get_var errors_near_es       es
		json_get_var errors_near_ses      ses
		json_get_var errors_near_fecs     fecs
		json_get_var errors_near_crc      crc_p
		json_get_var errors_near_crcp     crcp_p
		json_get_var errors_near_cv       cv_p
		json_get_var errors_near_cvp      cvp_p
		json_get_var errors_near_rtx_uc   rx_corrupted
		json_get_var errors_near_rtx_ucp  rx_uncorrected_protected
		json_get_var errors_near_rtx_rx   rx_retransmitted
		json_get_var errors_near_rtx_c    rx_corrected
		json_get_var errors_near_rtx_tx   tx_retransmitted
	json_close_object
	json_select far
		json_get_var errors_far_es        es
		json_get_var errors_far_ses       ses
		json_get_var errors_far_fecs      fecs
		json_get_var errors_far_crc       crc_p
		json_get_var errors_far_crcp      crcp_p
		json_get_var errors_far_cv        cv_p
		json_get_var errors_far_cvp       cvp_p
		json_get_var errors_far_rtx_uc    rx_corrupted
		json_get_var errors_far_rtx_ucp   rx_uncorrected_protected
		json_get_var errors_far_rtx_rx    rx_retransmitted
		json_get_var errors_far_rtx_c     rx_corrected
		json_get_var errors_far_rtx_tx    tx_retransmitted
	json_close_object
json_close_object

# present values to collectd
echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/uptime\" N:$uptime"
echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/bitrate-downstream\" N:$downstream_datarate"
echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/bitrate-downstream_max\" N:$downstream_datarate_max"
echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/snr-downstream\" N:$downstream_snr"
echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/bitrate-upstream\" N:$upstream_datarate"
echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/bitrate-upstream_max\" N:$upstream_datarate_max"
echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/snr-upstream\" N:$upstream_snr"
echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/errors-near_es\" N:$errors_near_es"
echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/errors-near_ses\" N:$errors_near_ses"
echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/errors-near_fecs\" N:$errors_near_fecs"
echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/errors-near_crc\" N:$errors_near_crc"
echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/errors-near_crcp\" N:$errors_near_crcp"
echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/errors-near_cv\" N:$errors_near_cv"
echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/errors-near_cvp\" N:$errors_near_cvp"
echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/errors-near_rtx_uc\" N:$errors_near_rtx_uc"
echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/errors-near_rtx_ucp\" N:$errors_near_rtx_ucp"
echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/errors-near_rtx_rx\" N:$errors_near_rtx_rx"
echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/errors-near_rtx_c\" N:$errors_near_rtx_c"
echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/errors-near_rtx_tx\" N:$errors_near_rtx_tx"
echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/errors-far_es\" N:$errors_far_es"
echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/errors-far_ses\" N:$errors_far_ses"
echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/errors-far_fecs\" N:$errors_far_fecs"
echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/errors-far_crc\" N:$errors_far_crc"
echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/errors-far_crcp\" N:$errors_far_crcp"
echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/errors-far_cv\" N:$errors_far_cv"
echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/errors-far_cvp\" N:$errors_far_cvp"
echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/errors-far_rtx_uc\" N:$errors_far_rtx_uc"
echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/errors-far_rtx_ucp\" N:$errors_far_rtx_ucp"
echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/errors-far_rtx_rx\" N:$errors_far_rtx_rx"
echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/errors-far_rtx_c\" N:$errors_far_rtx_c"
echo "PUTVAL \"$HOSTNAME/exec-lantiqdsl/errors-far_rtx_tx\" N:$errors_far_rtx_tx"

cat /www/luci-static/resources/statistics/rrdtool/definitions/exec.js

'use strict';
'require baseclass';
return baseclass.extend({
	title: _('DSL'),
	rrdargs: function(graph, host, plugin, plugin_instance, dtype) {
		var uptime = {
			title: "%H: DSL line uptime",
			vlabel: "days",
			alt_autoscale: true,
			number_format: "%5.1lf days",
			rrdopts: ["-h 80"],
			data: {
				types: ["uptime"],
				options: {
					uptime: {
						title:         "Uptime",
						transform_rpn: "86400,/",
						noavg:         true,
						color:         "007700"
					}
				}
			}
		};
		var datarate = {
			title: "%H: DSL datarates",
			vlabel: "bit/s",
			number_format: "%5.1lf %sbit/s",
			alt_autoscale: true,
			data: {
				instances: {
					bitrate: ["downstream", "downstream_max", "upstream", "upstream_max"]
				},
				options: {
					bitrate_downstream: {
						title:         "Downstream",
						color:         "007700"
					},
					bitrate_downstream_max: {
						title:         "Downstream (max.)",
						color:         "aaaaaa",
						overlay:       true,
						noarea:        true
					},
					bitrate_upstream: {
						title:         "Upstream",
						color:         "000077",
						flip:          true
					},
					bitrate_upstream_max: {
						title:         "Upstream (max.)",
						color:         "aaaaaa",
						overlay:       true,
						noarea:        true,
						flip:          true
					}
				}
			}
		};
		var snr = {
			title: "%H: DSL SNR",
			vlabel: "dB",
			number_format: "%4.1lf dB",
			alt_autoscale: true,
			data: {
				instances: {
					snr: ["downstream", "upstream"]
				},
				options: {
					snr_downstream: {
						title:         "Downstream",
						color:         "007700",
						overlay:       true
					},
					snr_upstream: {
						title:         "Upstream",
						color:         "000077",
						flip:          true
					}
				}
			}
		};
		var operational_counters = {
			title: "%H: DSL operational counters",
			vlabel: "events (per day)",
			y_min: -4000,
			y_max: 4000,
			alt_autoscale: true,
			data: {
				instances: {
					errors: ["near_fecs", "far_fecs"]
				},
				options: {
					errors_near_fecs: {
						title:         "FECs (near)",
						transform_rpn: "86400,*",
						color:         "007700"
					},
					errors_far_fecs: {
						title:         "FECs (far)",
						transform_rpn: "86400,*",
						color:         "000077",
						flip:          true
					}
				}
			}
		};
		var error_seconds = {
			title: "%H: DSL errored seconds",
			vlabel: "seconds (per day)",
			y_min: -200,
			y_max: 200,
			alt_autoscale: true,
			data: {
				instances: {
					errors: ["near_ses", "near_es", "far_ses", "far_es" ]
				},
				options: {
					errors_near_ses: {
						title:         "SES/severely errored seconds (near)",
						transform_rpn: "86400,*",
						color:         "ff0000"
					},
					errors_near_es: {
						title:         "ES/errored seconds (near)",
						transform_rpn: "86400,*",
						color:         "777777",
						overlay:       true,
						noarea:        true
					},
					errors_far_ses: {
						title:         "SES/severely errored seconds (far)",
						transform_rpn: "86400,*",
						color:         "ff0000",
						flip:          true
					},
					errors_far_es: {
						title:         "ES/errored seconds (upstream)",
						transform_rpn: "86400,*",
						color:         "777777",
						flip:          true,
						overlay:       true,
						noarea:        true
					}
				}
			}
		};
		var error_counters = {
			title: "%H: DSL error counters",
			vlabel: "errors (per day)",
			y_min: -200,
			y_max: 200,
			alt_autoscale: true,
			data: {
				instances: {
					errors: ["near_crc", "near_crcp", "far_crc", "far_crcp"]
				},
				options: {
					errors_near_crc: {
						title:         "CRC errors (near)",
						transform_rpn: "86400,*",
						color:         "444444"
					},
					errors_near_crcp: {
						title:         "CRC errors (preemptive, near)",
						transform_rpn: "86400,*",
						color:         "888888",
						overlay:       true,
						noarea:        true
					},
					errors_far_crc: {
						title:         "CRC errors (far)",
						transform_rpn: "86400,*",
						color:         "444444",
						flip:          true,
						overlay:       true,
						noarea:        true
					},
					errors_far_crcp: {
						title:         "CRC errors (preemptive, far)",
						transform_rpn: "86400,*",
						color:         "888888",
						flip:          true,
						overlay:       true,
						noarea:        true
					}
				}
			}
		};
		var retx_counters = {
			title: "%H: DSL G.INP(retx) retransmission counters",
			vlabel: "DTUs (per day)",
			y_min: -200,
			y_max: 200,
			alt_autoscale: true,
			data: {
				instances: {
					errors: ["near_rtx_uc", "near_rtx_ucp", "near_rtx_c", "near_rtx_tx", "far_rtx_uc", "far_rtx_ucp", "far_rtx_c", "far_rtx_tx"]
				},
				options: {
					errors_near_rtx_uc: {
						title:         "ReTx corrupted (near)",
						transform_rpn: "86400,*",
						color:         "880000",
						overlay:       true,
						noarea:        false
					},
					errors_near_rtx_ucp: {
						title:         "ReTx corrupted protected (near)",
						transform_rpn: "86400,*",
						color:         "ff0000",
						overlay:       true,
						noarea:        true
					},
					errors_near_rtx_c: {
						title:         "ReTx corrected (near)",
						transform_rpn: "86400,*",
						color:         "00ff00",
						overlay:       true,
						noarea:        true
					},
					errors_near_rtx_tx: {
						title:         "ReTx tx-retransmitted (near)",
						transform_rpn: "86400,*",
						color:         "ff00ff",
						overlay:       true,
						noarea:        true
					},
					errors_far_rtx_uc: {
						title:         "ReTx corrupted (far)",
						transform_rpn: "86400,*",
						color:         "880000",
						flip:          true,
						overlay:       true,
						noarea:        false
					},
					errors_far_rtx_ucp: {
						title:         "ReTx corrupted protected (far)",
						transform_rpn: "86400,*",
						color:         "ff0000",
						flip:          true,
						overlay:       true,
						noarea:        true
					},
					errors_far_rtx_c: {
						title:         "ReTx corrected (far)",
						transform_rpn: "86400,*",
						color:         "00ff00",
						flip:          true,
						overlay:       true,
						noarea:        true
					},
					errors_far_rtx_tx: {
						title:         "ReTx tx-retransmitted (far)",
						transform_rpn: "86400,*",
						color:         "ff00ff",
						flip:          true,
						overlay:       true,
						noarea:        true
					},
				}
			}
		};

		return [uptime,datarate,snr,operational_counters,error_seconds,error_counters,retx_counters];
	}
});

I have not yet tried any of the default to zero approaches, will update after testing. I do wonder whether using an undocumented feature in json_get_var is a robust way forward though, so I am more inclined to use the shell approach instead. If I do, would it make sense to just do this for all variables to reduce the dependency on the exact ubus dsl metrics?

Actually, I had a different thought. I believe there's a way to only create the graph if there's a corresponding RRD database, though I have to find out how it's done yet. If so, rather than zeroing out the values it would be more sensible to not submit those values to collectd at all if they are not available.

That sounds like a real plan :wink: forward.
But, ATM I am more concentrated on sanity checking the displayed data, I see events with more corrected DTUs than corrupted and retransmitted, which seems somewhat impossible, or (more likely) I have not yet fully understood what the counters are intending to tell me.

BUT, I do see that my upstream FECs correlate with ReTx events in the upstream, and also somewhat weaker in downstream direction (but there are no FECs reported for the downstream on my link). I have my doubts, that CPE in general are reliable in reporting these counters....

I guess it really is quite simple, the retransmissions are simply accounted on the other side, that is retransmissions for near corrupted DTU's appear on the far side's counters, which on second thought is pretty much where they should be accounted for :wink:
This pattern actually became clear only after I fixed the display of the counters. Now, I am pondering whether to leave things as they are now (and add a note to the description/legend) or whether I should flip the curves.... or both...

@takimata I have a question about the scaling values:

transform_rpn: "86400,*",

is number of seconds per day, but if I understand collectd's default configuration it samples every 30 seconds and calculates the difference to the last sample taken, which would be in samples/30 seconds, so should the conversion factor to sec/days not be 26024 = 2880 instead? Or does collectd convert to counts per second on sampling or storing to the rrd data-base?

It does. errors data sources (yes, data types are defined by the first word in the value name) are of the "DERIVE" data source type and end up as a "data rate" which is "calculated per second."

Sample frequency generally does not matter with collectd. It's been specifically designed to deal with infrequent and/or inconsistent update intervals in mind.

1 Like

Thanks, quite helpful....

So here is a version that has some internal consistency (values affecting the near side plotted up, values affecting the far side plotted down; area plots for retransmission events, as these potentially cause increased jitter/burstiness; area plots for rx_uncorrected_protected, as these should result in CRC events as well as these indicate true packet loss). I also reduced the scaling to per minute, because I believe for absolute difference values only the hour plot is useful and there per day leads to hard to interpret large values, for the larger time windows, I guess only the ratio of the different areas matter....

I have not even tried to tackle the make plotting depend on a metric being available on ubus.

cat exec.js:

'use strict';
'require baseclass';
return baseclass.extend({
	title: _('DSL'),
	rrdargs: function(graph, host, plugin, plugin_instance, dtype) {
		var uptime = {
			title: "%H: DSL line uptime",
			vlabel: "days",
			alt_autoscale: true,
			number_format: "%5.1lf days",
			rrdopts: ["-h 80"],
			data: {
				types: ["uptime"],
				options: {
					uptime: {
						title:         "Uptime",
						transform_rpn: "86400,/",
						noavg:         true,
						color:         "007700"
					}
				}
			}
		};
		var datarate = {
			title: "%H: DSL datarates",
			vlabel: "bit/s",
			number_format: "%5.1lf %sbit/s",
			alt_autoscale: true,
			data: {
				instances: {
					bitrate: ["downstream", "downstream_max", "upstream", "upstream_max"]
				},
				options: {
					bitrate_downstream: {
						title:         "Downstream",
						color:         "007700"
					},
					bitrate_downstream_max: {
						title:         "Downstream (max.)",
						color:         "aaaaaa",
						overlay:       true,
						noarea:        true
					},
					bitrate_upstream: {
						title:         "Upstream",
						color:         "000077",
						flip:          true
					},
					bitrate_upstream_max: {
						title:         "Upstream (max.)",
						color:         "aaaaaa",
						overlay:       true,
						noarea:        true,
						flip:          true
					}
				}
			}
		};
		var snr = {
			title: "%H: DSL SNR",
			vlabel: "dB",
			number_format: "%4.1lf dB",
			alt_autoscale: true,
			data: {
				instances: {
					snr: ["downstream", "upstream"]
				},
				options: {
					snr_downstream: {
						title:         "Downstream",
						color:         "007700",
						overlay:       true
					},
					snr_upstream: {
						title:         "Upstream",
						color:         "000077",
						flip:          true
					}
				}
			}
		};
		var operational_counters = {
			title: "%H: DSL operational counters",
			vlabel: "events (per day)",
			y_min: -4000,
			y_max: 4000,
			alt_autoscale: true,
			data: {
				instances: {
					errors: ["near_fecs", "far_fecs"]
				},
				options: {
					errors_near_fecs: {
						title:         "FECs (near)",
						transform_rpn: "86400,*",
						color:         "007700"
					},
					errors_far_fecs: {
						title:         "FECs (far)",
						transform_rpn: "86400,*",
						color:         "000077",
						flip:          true
					}
				}
			}
		};
		var error_seconds = {
			title: "%H: DSL errored seconds",
			vlabel: "seconds (per day)",
			y_min: -200,
			y_max: 200,
			alt_autoscale: true,
			data: {
				instances: {
					errors: ["near_ses", "near_es", "far_ses", "far_es" ]
				},
				options: {
					errors_near_ses: {
						title:         "SES/severely errored seconds (near)",
						transform_rpn: "86400,*",
						color:         "ff0000"
					},
					errors_near_es: {
						title:         "ES/errored seconds (near)",
						transform_rpn: "86400,*",
						color:         "777777",
						overlay:       true,
						noarea:        true
					},
					errors_far_ses: {
						title:         "SES/severely errored seconds (far)",
						transform_rpn: "86400,*",
						color:         "ff0000",
						flip:          true
					},
					errors_far_es: {
						title:         "ES/errored seconds (upstream)",
						transform_rpn: "86400,*",
						color:         "777777",
						flip:          true,
						overlay:       true,
						noarea:        true
					}
				}
			}
		};
		var error_counters = {
			title: "%H: DSL error counters",
			vlabel: "errors (per day)",
			y_min: -200,
			y_max: 200,
			alt_autoscale: true,
			data: {
				instances: {
					errors: ["near_crc", "near_crcp", "far_crc", "far_crcp"]
				},
				options: {
					errors_near_crc: {
						title:         "CRC errors (near)",
						transform_rpn: "86400,*",
						color:         "444444"
					},
					errors_near_crcp: {
						title:         "CRC errors (preemptive, near)",
						transform_rpn: "86400,*",
						color:         "888888",
						overlay:       true,
						noarea:        true
					},
					errors_far_crc: {
						title:         "CRC errors (far)",
						transform_rpn: "86400,*",
						color:         "444444",
						flip:          true,
						overlay:       true,
						noarea:        true
					},
					errors_far_crcp: {
						title:         "CRC errors (preemptive, far)",
						transform_rpn: "86400,*",
						color:         "888888",
						flip:          true,
						overlay:       true,
						noarea:        true
					}
				}
			}
		};
		var retx_counters = {
			title: "%H: DSL G.INP(retx) retransmission counters",
			vlabel: "DTUs (per minute)",
			y_min: -0.1,
			y_max: 0.1,
			alt_autoscale: true,
			data: {
				instances: {
					errors: ["far_rtx_tx", "near_rtx_uc", "near_rtx_ucp", "near_rtx_c", "near_rtx_tx", "far_rtx_uc", "far_rtx_ucp", "far_rtx_c"]
				},
				options: {
					errors_near_rtx_tx: {
						title:         "ReTx tx-retransmitted (far, accounted as near)",
						transform_rpn: "60,*",
						color:         "ff00ff",
						overlay:       false,
						flip:          true,
						noarea:        false
					},
					errors_near_rtx_uc: {
						title:         "ReTx corrupted (near)",
						transform_rpn: "60,*",
						color:         "ff0000",
						overlay:       true,
						noarea:        true
					},
					errors_near_rtx_c: {
						title:         "ReTx corrected (near)",
						transform_rpn: "60,*",
						color:         "00ff00",
						overlay:       true,
						noarea:        true
					},
					errors_near_rtx_ucp: {
						title:         "ReTx corrupted protected (near)",
						transform_rpn: "60,*",
						color:         "0000ff",
						overlay:       true,
						noarea:        false
					},
					errors_far_rtx_tx: {
						title:         "ReTx tx-retransmitted (near, accounted as far)",
						transform_rpn: "60,*",
						color:         "af00af",
						overlay:       false,
						noarea:        false
					},
					errors_far_rtx_uc: {
						title:         "ReTx corrupted (far)",
						transform_rpn: "60,*",
						color:         "af0000",
						flip:          true,
						overlay:       true,
						noarea:        true
					},
					errors_far_rtx_c: {
						title:         "ReTx corrected (far)",
						transform_rpn: "60,*",
						color:         "00af00",
						flip:          true,
						overlay:       true,
						noarea:        true
					},
					errors_far_rtx_ucp: {
						title:         "ReTx corrupted protected (far)",
						transform_rpn: "60,*",
						color:         "0000af",
						flip:          true,
						overlay:       true,
						noarea:        false
					},
				}
			}
		};

		return [uptime,datarate,snr,operational_counters,error_seconds,error_counters,retx_counters];
	}
});

I can't say much about the rtx graphs, I don't have them, so I trust you that they make sense. Honestly, I'm agnostic when it comes to the scaling factor, if "per hour" makes sense to debug, then per hour it is. I just know that either you leave it per-second to get meaningful totals, or scale them per-anything-else to get meaningful averages.

Thank you though for looking into the meaning of the "preemptive CRC errors", I had the suspicion that they were part of the total CRC errors, but heck if I could find any definitive answer on that. (And empirical observation is hard for me, after a month of uptime I am still looking at a zero counter.) So overlaying those makes sense.

As for the additional graps for rtx being there at all, I think the key is in per-instance definitions, but I have to wrap my head around rrdtool.js first and do some tests.

Also I am moving the scripts to a proper Github repository, I'm a bit tired of the Gists that have outlived their usefulness. That would also make it possible for you to directly submit a PR. (Edit:) or I'll pass the baton to someone else entirely, I am the farthest from claiming I am the official instance for this project.

1 Like

;), I think by iterative trial and error they should be okay now :wink:

Fair enough, makes sense, with a sample interval of ~30 seconds a single event will be at 1/30 = 0.033 Hz, which I guess is okay, similar to per minute... per day just leads to a "quantization" with really large and unintuitive steps in the hour plot...

Well, that is pretty opaque, from what I see Rec. ITU-T G.992.3 (04/2009) Annex N mentions the options preemption (Annex N.3.1.2 Support for preemption) and short packages (Annex N.3.1.3 Support for short packets), which is referenced in Rec. ITU-T G.993.2 (02/2019) at three places with terse references back to 992.3. So my interpretation is just my best interpretation of the data I have found (that does not contradict the ITU standrads as far as I can see), and not strictly backed up by explicit text....

This at least fits with the expectation that no-one actually uses this... on ADSL2 (G992.3) with low bitrates pre-emption might have made sense (to allow VoIP packets to get real piority access to the link) but with VDSL2 and the typical bitrates it might simply not be used at all, at which point zero counters are expected, no?

Whatever you prefer, I can see the whole PR/github thing as a helpful formalization, but I am not sure whether the DSL collectd module will actually see much more changes at a high enough rate (then, I naively would probably try to package everything as a repository anyways ;))

I am considerably less qualified than you, and will in all likely hood switch back to the Zyxel after I helped out a bit with the lantiq* (as the lantiq is still fickle as hell compared to the Zyxel with multiple unenforced retrains during my short testing period; yes due to my wiring fix-up it seems to be better than in the past, but the Zyxel kept sync for months on end, while with the HH5A it days).

*) I will give it a bit ore time, because my wildest error type looked like the vectoring engine stopped working, which might be fixed by actually sending error samples back to the VE, so that under changing conditions in the binder and hence changing cross-talk patterns vectoring might be able to adapt; but that error was only sporadic to begin with so....

Maybe, but that's not what I mean. I'm literally looking at zero CRC errors on my line and therefore cannot make any meaningful observations:


I therefore need to rely on other people's input.

Tangent: I'm sorry to hear that the Lantiq modem is still temperamental on your line, which by means that it would probably not do much better on my home line either. I will probably not switch away from my Zyxel there. And once international travel is possible again I'm planning to emigrate in the following few months, to a country where the optic fibres grow plentiful. So there's an expiration date on my DSL adventures anyway.

No you're not.

well it works on my HH5

line is stable but i guess has some noise now and then. When i moved in over a year ago i got 80/20 reported. now its down to 51/10 ish. Enough that i know not to bother paying for anything more than 40/10

That is curious, lots of errored seconds but almost no CRC errors that would account for them. I wonder what the underlying cause for the ES is.

Ah, good point, and nice link :wink:

If I find a trick to make it stable I will post it here, but so far it is not much better than in the past, so the issue was neither the then high error rates on my link, nor the fact that vectoring error samples could not be sent... (or by using a more recent firmware blob I might now be running into totally new issues). As I said, I will likely revert back to the Zyxel, sure OpenWrt is nice as a modem OS, but it is even nicer not having to worry at all about a modem :wink:

In that case, a safe trip and enjoy!

Regarding collectd and exec scripts it is not even a competition, I am not even playing the same game :wink:

from the other thread on vectoring Vectoring on Lantiq VRX200 / VR9 - missing callback for sending error samples

https://xdarklight.github.io/lantiq-xdsl-firmware-info/ < dsl firmware versions if u want to play with other versions.

1 Like

Thanks, I have been there before :wink: turns out for me the dm200 5.8.1.5.0.7 (dsl_vr9_firmware_xdsl-05.08.01.05.00.07_05.08.00.09.00.01.bin) blob goes into a silent->handshake->fullinit->silent loop not sure why...

Thanks for these scripts - saved me time and taught me loads (I think!).
I have been using them for a few days and I may have found a solution to the STDERR problem - but I am a complete noob, so take what I say with a pinch of salt!

In exec-lantiqdsl.sh after you set the hostname I have set an interval and then wrapped the rest of the script (not the exit) in a function.
At the bottom, after the function end (but before the exit) I added a "while true" loop where I call the function I just wrapped the script in and also sleep for the interval. Obviously, the exit is now superfluous!
I got this idea from the SQM collectd scripts... I believe collectd/exec takes care not to create duplicate processes - and I read somewhere this is more efficient than making collectd/exec create a new process every time (but I seriously have no clue!).

At the same time as doing this I also removed some of the data collection (CRC's and FECS) that I wasn't interested in and added HEC errors - so I suppose it's possible this "loop" idea does nothing, and my other changes somehow got rid of the STDERR error.

Although, when I removed the while loop from the SQM script it too triggered a random STDERR daemon error from that script! Soooo... yeah!

My suspicion is that its something to do with STDERR buffering its writes.
Instead of my "loop" solution, maybe flushing STDERR before exiting the script would also work?
I haven't tried - mostly because I'm clueless about how to do it!

Feel free to point out any mistakes, bad practice, poor nomenclature... or if all of this is rubbish please just remove the post - no hard feelings!

1 Like

Quite the contrary, thank you for the input!

I am aware that this is one of the two possibilities:

  • have collectd run the script periodically
  • have the script run once in an endless loop and report values periodically

Functionally, both options do pretty much the same thing. With the latter option, of course it will never result in that STDERR error simply because the script never ends and never returns any exit codes or errors to collectd. However, it will also run "asynchronously" besides collectd, and to date I have not found any indication which one of the two options is more appropriate.

I will look into it. I have another router to convert to 21.02 soon, with a different script for a different modem, but doing the same thing, and I will take another stab at it.

1 Like