Sweet. Any chance you could push those fixes? I've finished the tsping wrapper for cake-autorate and dying to test this out!
Please use stdbuf -oL as a wrapper as a temporary workaround. To solve the problem properly, please add a call to setvbuf() to the C code.
1 Like
Why does stdbuf -oL change the operation of tsping in this way? See:
rroot@OpenWrt-1:~/cake-autorate# stdbuf -oL tsping --print-timestamps --machine-readable=' ' --sleep-time 0 --target-spacing 300 9.9.9.9
9.9.9.10 9.9.9.11 185.228.168.9 185.228.168.10
Starting tsping 0.2.2 - pinging 5 targets
1678121889.858740 9.9.9.9 0 61089809 61089834 61089834 61089858 49 24 25
Something went wrong while sending: -1
1678121890.457717 9.9.9.11 0 61090409 61090433 61090433 61090457 48 24 24
1678121890.763506 185.228.168.9 0 61090710 61090737 61090737 61090763 53 26 27
1678121891.059733 185.228.168.10 0 61091010 61091035 61091035 61091059 49 24 25
1678121891.358621 9.9.9.9 1 61091310 61091333 61091333 61091358 48 25 23
Something went wrong while sending: -1
1678121891.957635 9.9.9.11 1 61091911 61091933 61091933 61091957 46 24 22
1678121892.258752 185.228.168.9 1 61092212 61092236 61092236 61092258 46 22 24
1678121892.558668 185.228.168.10 1 61092512 61092534 61092534 61092558 46 24 22
1678121892.857691 9.9.9.9 2 61092812 61092833 61092833 61092857 45 24 21
Something went wrong while sending: -1
1678121893.457572 9.9.9.11 2 61093413 61093433 61093433 61093457 44 24 20
1678121893.758712 185.228.168.9 2 61093713 61093737 61093737 61093758 45 21 24
1678121894.059684 185.228.168.10 2 61094013 61094035 61094035 61094059 46 24 22
1678121894.357632 9.9.9.9 3 61094313 61094333 61094333 61094357 44 24 20
Something went wrong while sending: -1
^C
root@OpenWrt-1:~/cake-autorate# tsping --print-timestamps --machine-readable=' ' --sleep-time 0 --target-spacing 300 9.9.9.9 9.9.9.10 9.
9.9.11 185.228.168.9 185.228.168.10
Starting tsping 0.2.2 - pinging 5 targets
1678122015.593081 9.9.9.9 0 61215543 61215564 61215564 61215593 50 29 21
1678122015.899172 9.9.9.10 0 61215843 61215871 61215871 61215899 56 28 28
1678122016.201457 9.9.9.11 0 61216144 61216175 61216175 61216201 57 26 31
1678122016.507425 185.228.168.9 0 61216444 61216481 61216481 61216507 63 26 37
1678122016.792759 185.228.168.10 0 61216744 61216768 61216768 61216792 48 24 24
1678122017.100950 9.9.9.9 1 61217045 61217076 61217076 61217100 55 24 31
1678122017.403393 9.9.9.10 1 61217345 61217374 61217374 61217403 58 29 29
1678122017.739296 9.9.9.11 1 61217645 61217711 61217711 61217739 94 28 66
1678122017.989428 185.228.168.9 1 61217945 61217967 61217967 61217989 44 22 22
1678122018.295854 185.228.168.10 1 61218245 61218268 61218268 61218295 50 27 23
1678122018.592690 9.9.9.9 2 61218546 61218566 61218566 61218592 46 26 20
1678122018.919731 9.9.9.10 2 61218846 61218895 61218895 61218919 73 24 49
1678122019.205083 9.9.9.11 2 61219146 61219171 61219171 61219205 59 34 25
Yes I will get those fixes up asap
Getting the columns right, would be simplest to fix by changing them to have the same ordering, so let me know if there's any preferences on that
That is really weird, and seems to only affect 9.9.9.10. I've neglected to use errno if something goes wrong when sending, so will add that as well so we can better see what it's complaining about
Regarding time interval, fping uses:
−p, −−period= MSEC
In looping or counting modes (−l, −c, or −C), this parameter sets the time in milliseconds that fping waits between successive packets to an individual target. Default is 1000 and minimum is 10.
−i, −−interval= MSEC
The minimum amount of time (in milliseconds) between sending a ping packet to any target (default is 10, minimum is 1).
So with:
fping ${ping_extra_args} --timestamp --loop --period "${reflector_ping_interval_ms}" --interval "${ping_response_interval_ms}" --timeout 10000 "${reflectors[@]:0:${no_pingers}}"
As I understand it, this means that ${reflector_ping_interval_ms} separates ping sends to a particular target, and ${ping_response_interval_ms} separates ping sends to any target.
I think this system in fping probably is optimal. @patrakov do you agree?
@moeller0 finally - OWDs in cake-autorate using @Lochnair's tsping binary.
@patrakov would you like to test?
Samples per reflector:
ReflectorID: 185.228.168.10; N: 91
ReflectorID: 185.228.168.9; N: 91
ReflectorID: 9.9.9.11; N: 90
ReflectorID: 9.9.9.9; N: 88
DL: maximum 95.000%-ile delta delay over all 4 reflectors: 18.670 ms.
DL: maximum 99.000%-ile delta delay over all 4 reflectors: 38.850 ms.
DL: maximum 99.500%-ile delta delay over all 4 reflectors: 38.850 ms.
DL: maximum 99.900%-ile delta delay over all 4 reflectors: 38.850 ms.
DL: maximum 99.950%-ile delta delay over all 4 reflectors: 38.850 ms.
DL: maximum 99.990%-ile delta delay over all 4 reflectors: 38.850 ms.
DL: maximum 99.999%-ile delta delay over all 4 reflectors: 38.850 ms.
UL: maximum 95.000%-ile delta delay over all 4 reflectors: 4.990 ms.
UL: maximum 99.000%-ile delta delay over all 4 reflectors: 11.985 ms.
UL: maximum 99.500%-ile delta delay over all 4 reflectors: 11.985 ms.
UL: maximum 99.900%-ile delta delay over all 4 reflectors: 11.985 ms.
UL: maximum 99.950%-ile delta delay over all 4 reflectors: 11.985 ms.
UL: maximum 99.990%-ile delta delay over all 4 reflectors: 11.985 ms.
UL: maximum 99.999%-ile delta delay over all 4 reflectors: 11.985 ms.
INFO: Writing plot as: ./output.timecourse.pdf
INFO: fn_parse_autorate_log took: 16.0685 seconds.
1 Like
Not necessarily today, as the LTE link is good at night (20+ Mbps) and doesn't really need cake-autorate. And I won't have anything demanding (like Discord) until Wednesday.
Regarding mimicking fping options - yes I agree that it makes sense. That's assuming a sane interpretation of the timeout, in particular the case when it is greater than the period.
1 Like
Why? It is not that fping is a widely used binary with lots of documentation...
also does fping allow period smaller than interval?
Yes it does:
hms-beagle2:~ smoeller$ fping --timestamp --loop --period 100 --interval 200 1.1.1.1 9.9.9.9
[1678125449.51688] 1.1.1.1 : [0], 64 bytes, 33.5 ms (33.5 avg, 0% loss)
[1678125449.71822] 9.9.9.9 : [0], 64 bytes, 32.9 ms (32.9 avg, 0% loss)
[1678125449.92214] 1.1.1.1 : [1], 64 bytes, 33.0 ms (33.2 avg, 0% loss)
[1678125450.12517] 9.9.9.9 : [1], 64 bytes, 33.3 ms (33.1 avg, 0% loss)
[1678125450.33064] 1.1.1.1 : [2], 64 bytes, 33.6 ms (33.4 avg, 0% loss)
[1678125450.53283] 9.9.9.9 : [2], 64 bytes, 33.6 ms (33.3 avg, 0% loss)
[1678125450.73605] 1.1.1.1 : [3], 64 bytes, 33.7 ms (33.4 avg, 0% loss)
[1678125450.93910] 9.9.9.9 : [3], 64 bytes, 33.4 ms (33.3 avg, 0% loss)
^C
however the results are "interesting" at best... tspings variant is IMHO better in that you can only specify things the code will deliver...
Normally I am all for re-using existing command line switches and behaviour, but here I am not convinced that fping is the example to follow.
1 Like
I like how apparently both autorate itself as well as the plotting code defaulted to do the right thing once fed with OWDs.
This will need testing and likely there will be one or two issues to iron out:
But it at least allowed me to produce the plot above using @Lochnair's tsping binary, which is available here:
and which includes support for ICMP type 13 (timestamp) requests and responses, and hence working with one way delays (OWDs) rather than round trip times (RTTs).
tsping is not yet an official OpenWrt package, but for anyone wanting to test there are simple instructions for building on OpenWrt here:
For any readers not sure about the significance of this, this facilitates determining the direction of bufferbloat (download or upload) and hence better control over the download and upload shaper rates since they can be controlled more independently in dependence upon the respective download and upload OWDs. Hitherto cake-autorate has fudged the issue by working with RTTs and erred on the side of caution.
Here is an example set of config overrides that work for my connection:
pinger_binary=tsping
reflectors=(
"9.9.9.9" "9.9.9.10" "9.9.9.11" # Quad9
"185.228.168.9" "185.228.168.10" # CleanBrowsing
)
no_pingers=5
ping_prefix_string="stdbuf -oL"
But does this make sense to you:
# tsping --help
Usage: tsping [OPTION...] IP1 IP2 IP3 ...
tsping -- a simple application to send ICMP echo/timestamp requests
-e, --icmp-echo Use ICMP echo requests
-t, --icmp-ts Use ICMP timestamp requests (default)
-r, --target-spacing=TIME Time to wait between pinging each target in ms
(default 0)
-s, --sleep-time=TIME Time to wait between each round of pinging in ms
(default 100)
-D, --print-timestamps Print UNIX timestamps for responses
-m, --machine-readable[=DELIMITER]
Output results in a machine readable format
-f, --fw-mark=MARK Firewall mark to set on outgoing packets
-i, --interface=INTERFACE Interface to bind to
-?, --help Give this help list
--usage Give a short usage message
Mandatory or optional arguments to long options are also mandatory or optional
for any corresponding short options.
With the above, my understanding is that sleep-time 1000ms and target-spacing 500ms means: every 1000ms start round robin and send out first ICMP to first target, and then wait 500ms to send out interval to next target in that round robin.
So effective interval between responses of 500ms.
But I see:
root@OpenWrt-1:~# tsping --print-timestamps --machine-readable=' ' --sleep-time 1000 --target-spacing 500 9.9.9.9 9.9.9.10
Starting tsping 0.2.2 - pinging 2 targets
1678128598.015933 9.9.9.10 0 67797969 67797993 67797993 67798015 46 22 24
1678128599.535897 9.9.9.9 1 67799470 67799504 67799504 67799535 65 31 34
1678128600.016869 9.9.9.10 1 67799970 67799994 67799994 67800016 46 22 24
1678128601.515681 9.9.9.9 2 67801471 67801494 67801494 67801515 44 21 23
1678128602.017769 9.9.9.10 2 67801971 67801993 67801993 67802017 46 24 22
1678128603.516854 9.9.9.9 3 67803471 67803493 67803493 67803516 45 23 22
1678128604.015840 9.9.9.10 3 67803972 67803993 67803993 67804015 43 22 21
That is, 1.5s and 500ms spacing between responses.
This is much clearer to me:
−p, −−period= MSEC
In looping or counting modes (−l, −c, or −C), this parameter sets the time in milliseconds that fping waits between successive packets to an individual target. Default is 1000 and minimum is 10.
−i, −−interval= MSEC
The minimum amount of time (in milliseconds) between sending a ping packet to any target (default is 10, minimum is 1).
Nope, with e.g. tow IPs, you will do:
probe1, 500ms, probe2, 1000ms
so your total period is 1500ms...
And that is what you see, 500ms between the members of the set (9.9.9.9's first response is missing)
- 9.9.9.9: 0
- 9.9.9.10: 500ms later
next round - 9.9.9.9: 1000ms after 2), so exactly the requested sleep time.
Except you can specify stuff that fping does not complain about and does not deliver, period < interval...
My take is that you got used to fpings way of specifying this and hence confuse sleep with period, even though these are clearly different concepts. What like about @Lochnair's version is that you can not request impossible timings... it does become a tiny bit more involved to calculate the effective per reflector rate.
1 Like
Ah, I think I get it now. So they are both spacings between sends. Target is the spacing between targets, and sleep is the spacing between rounds (i.e. the spacing between the last send and the next first send)?
Ah, not quite:
root@OpenWrt-1:~# tsping --print-timestamps --machine-readable=' ' --sleep-time 1000 --target-spacing 1000 9.9.9.9 9.9.9.10
Starting tsping 0.2.2 - pinging 2 targets
1678131109.591224 9.9.9.9 0 70309547 70309563 70309563 70309591 44 28 16
1678131110.598166 9.9.9.10 0 70310547 70310574 70310574 70310598 51 24 27
1678131112.597135 9.9.9.9 1 70312547 70312573 70312573 70312597 50 24 26
1678131113.598123 9.9.9.10 1 70313548 70313573 70313573 70313598 50 25 25
1678131115.609165 9.9.9.9 2 70315548 70315584 70315584 70315609 61 25 36
1678131116.598139 9.9.9.10 2 70316548 70316573 70316573 70316598 50 25 25
So there is target + sleep between rounds? But it should just be sleep right? Or?
Ah, now I understand your question @Lochnair:
Yes, I think it should be skipped. @moeller0 do you agree?
Good question, I guessthe question is convenience, do you want to be able to specify a "normal" continous sampling with just one number, or do you essentially always want having to configure sleep >= target. I honestly see only two usecases:
a) equitemporal sampling like autorate does now
b) batched sampling where you queuery with target 0 and then use sleep to define the effective period
since we mostly use a) I am fine with not having to configure sleep, but the behaviour and help text should be adjusted to:
-s, --sleep-time=TIME Time to wait between each round of pinging in ms
(default to target for equidistant sampling)
I have already filed a bug with a different (more consistent) proposal: https://github.com/Lochnair/tsping/issues/7
@patrakov any clue why this happens:
root@OpenWrt-1:~# tsping --print-timestamps --machine-readable=' ' --sleep-time 100 --target-spacing 1000 9.9.9.10 9.9.9.9
Starting tsping 0.2.2 - pinging 2 targets
1678132116.263362 9.9.9.10 0 71316226 71316239 71316239 71316263 37 24 13
1678132118.388443 9.9.9.10 1 71318327 71318364 71318364 71318388 61 24 37
1678132119.377351 9.9.9.9 1 71319327 71319353 71319353 71319377 50 24 26
1678132120.490509 9.9.9.10 2 71320428 71320464 71320464 71320490 62 26 36
root@OpenWrt-1:~# stdbuf -oL tsping --print-timestamps --machine-readable=' ' --sleep-time 100 --target-spacing 1000 9.9.9.10 9.9.9.9
Starting tsping 0.2.2 - pinging 2 targets
1678132123.268491 9.9.9.10 0 71323218 71323244 71323244 71323268 50 24 26
Something went wrong while sending: -1
1678132125.367498 9.9.9.10 1 71325318 71325343 71325343 71325367 49 24 25
Something went wrong while sending: -1
1678132127.468529 9.9.9.10 2 71327419 71327443 71327443 71327468 49 25 24
Something went wrong while sending: -1
For some reason the address family is corrupted:
[pid 26979] sendto(3, "\r\0\346\253ia\0\0\4B\236\260\0\0\0\0\0\0\0\0", 20, 0, {sa_family=0x7c60 /* AF_??? */, sa_data="\36\205\t\t\t\t\270\352'\205\177\0\0\0"}, 16) = -1 EAFNOSUPPORT (Address family not supported by protocol)
I have compiled it on the desktop, and valgrind complains:
==25044== Thread 3:
==25044== Syscall param socketcall.sendto(to.sa_family) points to uninitialised byte(s)
==25044== at 0x498DEAC: sendto (sendto.c:27)
==25044== by 0x10AB28: send_icmp_timestamp_request (main.c:94)
==25044== by 0x10B60C: sender_loop (main.c:304)
==25044== by 0x4909BB4: start_thread (pthread_create.c:444)
==25044== by 0x498BCB3: clone (clone.S:100)
==25044== Address 0x4a6e3d0 is 0 bytes inside a block of size 32 alloc'd
==25044== at 0x4846CC3: realloc (vg_replace_malloc.c:1437)
==25044== by 0x10A76B: parse_opt (args.h:103)
==25044== by 0x499846C: group_parse (argp-parse.c:257)
==25044== by 0x499846C: parser_parse_arg (argp-parse.c:693)
==25044== by 0x499846C: parser_parse_next (argp-parse.c:865)
==25044== by 0x499846C: argp_parse (argp-parse.c:921)
==25044== by 0x10B747: main (main.c:341)
1 Like
Simply do not do this, white space is arguable the worst possible delimiter use something explicit here like ; that will allow you to detect empty fields. All you need to do is something
like:
orig_IFs=${IFS}
IFS=";" read -t ...
IFS=${orig_IFS}
to have read use proper delimiters...