This will need testing and likely there will be one or two issues to iron out:
But it at least allowed me to produce the plot above using @Lochnair's tsping binary, which is available here:
and which includes support for ICMP type 13 (timestamp) requests and responses, and hence working with one way delays (OWDs) rather than round trip times (RTTs).
tsping is not yet an official OpenWrt package, but for anyone wanting to test there are simple instructions for building on OpenWrt here:
For any readers not sure about the significance of this, this facilitates determining the direction of bufferbloat (download or upload) and hence better control over the download and upload shaper rates since they can be controlled more independently in dependence upon the respective download and upload OWDs. Hitherto cake-autorate has fudged the issue by working with RTTs and erred on the side of caution.
Here is an example set of config overrides that work for my connection:
# tsping --help
Usage: tsping [OPTION...] IP1 IP2 IP3 ...
tsping -- a simple application to send ICMP echo/timestamp requests
-e, --icmp-echo Use ICMP echo requests
-t, --icmp-ts Use ICMP timestamp requests (default)
-r, --target-spacing=TIME Time to wait between pinging each target in ms
(default 0)
-s, --sleep-time=TIME Time to wait between each round of pinging in ms
(default 100)
-D, --print-timestamps Print UNIX timestamps for responses
-m, --machine-readable[=DELIMITER]
Output results in a machine readable format
-f, --fw-mark=MARK Firewall mark to set on outgoing packets
-i, --interface=INTERFACE Interface to bind to
-?, --help Give this help list
--usage Give a short usage message
Mandatory or optional arguments to long options are also mandatory or optional
for any corresponding short options.
With the above, my understanding is that sleep-time 1000ms and target-spacing 500ms means: every 1000ms start round robin and send out first ICMP to first target, and then wait 500ms to send out interval to next target in that round robin.
That is, 1.5s and 500ms spacing between responses.
This is much clearer to me:
−p, −−period= MSEC
In looping or counting modes (−l, −c, or −C), this parameter sets the time in milliseconds that fping waits between successive packets to an individual target. Default is 1000 and minimum is 10.
−i, −−interval= MSEC
The minimum amount of time (in milliseconds) between sending a ping packet to any target (default is 10, minimum is 1).
Nope, with e.g. tow IPs, you will do:
probe1, 500ms, probe2, 1000ms
so your total period is 1500ms...
And that is what you see, 500ms between the members of the set (9.9.9.9's first response is missing)
9.9.9.9: 0
9.9.9.10: 500ms later
next round
9.9.9.9: 1000ms after 2), so exactly the requested sleep time.
Except you can specify stuff that fping does not complain about and does not deliver, period < interval...
My take is that you got used to fpings way of specifying this and hence confuse sleep with period, even though these are clearly different concepts. What like about @Lochnair's version is that you can not request impossible timings... it does become a tiny bit more involved to calculate the effective per reflector rate.
Ah, I think I get it now. So they are both spacings between sends. Target is the spacing between targets, and sleep is the spacing between rounds (i.e. the spacing between the last send and the next first send)?
Good question, I guessthe question is convenience, do you want to be able to specify a "normal" continous sampling with just one number, or do you essentially always want having to configure sleep >= target. I honestly see only two usecases:
a) equitemporal sampling like autorate does now
b) batched sampling where you queuery with target 0 and then use sleep to define the effective period
since we mostly use a) I am fine with not having to configure sleep, but the behaviour and help text should be adjusted to:
-s, --sleep-time=TIME Time to wait between each round of pinging in ms
(default to target for equidistant sampling)
[pid 26979] sendto(3, "\r\0\346\253ia\0\0\4B\236\260\0\0\0\0\0\0\0\0", 20, 0, {sa_family=0x7c60 /* AF_??? */, sa_data="\36\205\t\t\t\t\270\352'\205\177\0\0\0"}, 16) = -1 EAFNOSUPPORT (Address family not supported by protocol)
I have compiled it on the desktop, and valgrind complains:
==25044== Thread 3:
==25044== Syscall param socketcall.sendto(to.sa_family) points to uninitialised byte(s)
==25044== at 0x498DEAC: sendto (sendto.c:27)
==25044== by 0x10AB28: send_icmp_timestamp_request (main.c:94)
==25044== by 0x10B60C: sender_loop (main.c:304)
==25044== by 0x4909BB4: start_thread (pthread_create.c:444)
==25044== by 0x498BCB3: clone (clone.S:100)
==25044== Address 0x4a6e3d0 is 0 bytes inside a block of size 32 alloc'd
==25044== at 0x4846CC3: realloc (vg_replace_malloc.c:1437)
==25044== by 0x10A76B: parse_opt (args.h:103)
==25044== by 0x499846C: group_parse (argp-parse.c:257)
==25044== by 0x499846C: parser_parse_arg (argp-parse.c:693)
==25044== by 0x499846C: parser_parse_next (argp-parse.c:865)
==25044== by 0x499846C: argp_parse (argp-parse.c:921)
==25044== by 0x10B747: main (main.c:341)
Simply do not do this, white space is arguable the worst possible delimiter use something explicit here like ; that will allow you to detect empty fields. All you need to do is something
like:
This was a general remark about not using ' ' or '/tab' as delimiters... delimiters need to be human readable and allow for unambiguous encoding of empty fields.
Tested on desktop and on the router, this gets rid of the error.
@Lochnair could you please release a fixed version? I don't care for #7 (sleep duration syntax) right now, but uninitialized variables are important. And this is a good time to add the setbuf() call.
I had a go at one of your irtt data sets (with octave*) and I am probably not understanding the data all that well, but could it be that the clocks of both sides are badly synchronized and both drifting? That, or I am looking at the wrong fields of irtt's json output
*) I use matlab at work so this is the only thing I am somewhat fluent in, but boy compared to commercial matlab, octave is a bumpy ride...
P.S.: The json parsing is still dog-slow, but you only need to do this once per input file, I just needed to find a loading function that actually works, and then massage irtt's output into a simple 2D table to make plotting simple, but since that is now done, refining the plot should not be witchcraft.
P.P.S.: Here is the current state of the plotting:
Since the clocks are out of sync this only shows the RTT, but adds the inter-quartile mean (the number we assume Ookla's speedtests.net reports) and the number of lost probes separated by direction. What is missing are for each of rtt/send/receive CDF and distribution plots. Unlike with the autorate data we have no achieved throughput or CPU load numbers to show concurrently... However irtt timestamps appear to be unix epoch in nanoseconds, so this could be co-plotted with autorate data containing throughput data....
Really neat that you've written this. I'm not aware of any decent ping binary that offers round robin ICMP type 13 requests. So a nice addition to the OpenWrt landscape.
I saw that same drift in the Julia code. Julia parses the json virtually instantaneously (once the initial call which involves a compile is done) so you can easily plot many datasets. The first plot takes 23s (to load all the packages). The next plot takes 0.12 s to read the data and make the plot.
Julia 1.9 is about to come out and it should dramatically lower startup times. I would guess maybe 3-5 seconds to start up based on the good stuff I'm hearing about 1.9.
I didn't spend much time designing a good plot visualization, and I find the offsets confusing... what is -400 ms delay (the red line) what is +500ms (green line)? What are those measured relative to? I think probably for interpretability they should have some offset added to them, but I don't know what. if someone can figure out what is going on there I'm happy to do the calcs though.
It kind of looks to me like rtt = send + receive, why receive is negative I don't know.
That is because one way delay measurements require synchronized clocks...
Let's assume a true RTT of 100ms on a symmetric path so both OWDs equal 50ms:
client clock 1 second ahead of server
client sends packet at local time 0, which is time -1000 at the server
packet arrives at server at 50ms client time, but the server stamps it with receive time -950ms and sends it back with send timestamp -951(1ms local processing time)
For true symmetric paths one can try to assume path delay to be symmetric as well and try to correct for it (NTP does that under the hood) but that is suboptimal if done during load tests like the speedtest here where the link experiences variable directional queueing delay, violating the equality assumption above....
