According to my tests, there are three processes involved: test.sh (parent), test.sh (child), and ping. The child seems to be a subshell, but I don't know why it appears here. Anyway all that it does is to wait for the ping process to finish.
procd sends the SIGTERM signal only to the parent process, which indeed terminates. Therefore, the pipe between it and the ping process gets severed from the reader (receiver) end. ping tries to write something there, and, because it is writing to a pipe the other end of which is closed, it gets a SIGPIPE.
Without procd, this signal terminates ping, and then the child test.sh process also finishes waiting for ping to exit, and exits itself, leaving nothing.
With procd, SIGPIPE is ignored, and therefore ping doesn't die, and the test.sh child process continues waiting, in vain. Therefore, two processes remain.
Bad news: in bash, there is no way to restore the "normal" handling of SIGPIPE, this is even documented in the manual page:
Signals ignored upon entry to the shell cannot be trapped or reset.
Terrific analysis! Is there by any chance a way to disable the ignoring of SIGPIPE in procd or systemd?
In any case, it seems that perhaps the safest option is to go on retaining the PID and explicitly killing it even though it's less elegant than the pipe teardown mechanism.
But you now can encapsulate the whole pinger PID locally inside the parser process an nobody else needs to know, you just keep a single PID. seems easy enough and avoids the need for elaborate process management...
the main loop only needs the maintain and logger PIDs, and maintain the parser PIDs and each parser handles a single binary.... seems pretty clean to me clean enough to be able to not need process management...
I think we can take some inspiration from mwan3: it is also implemented in shell, runs ping and other subprocesses, and is forced to run in an environment where SIGPIPE does not work (which is, in my opinion, an unsupported environment for any shell).
It's been a while since i posted in this thread. life and all that jazz. I have been using the new bash implementation ever since the Lua stopped to work due to reasons already explained. As a user seeking solutions, it's a luxury to have these options. As far as i can determine, the recent bash version does seem to do what i want it to do. But you guys need data. I would love to serve up some here next weekend.
It would be great if compatibility with other Linux distros could be preserved, as it seems to be the case so far in my x86 Debian setup.
But please, I don't want to be disrespectful or abusive in any way, asking something like this in an ... OpenWrt forum!
Of course you are OpenWrt devs thus your main goal is to produce OpenWrt-tailored code.
Anyway, thanks for the good work so far, and thanks for not kicking me out even if I'm not an OpenWrt user. (I still have OpenWrt in a wifi access point device, though )
If that is true then there is our path forward, trap that term and then initiate a proper staged shutdown, preferably by first asking each process nicely to shut itself down gracefully followed by a SIGKILL (and I mean -9 here the time to ask politely is when sending the shutdown request, so the "talk softly, but carry a big stick" of process management, if you will)
My plan is that each process creates a FIFO and reads that, so all information for that process should be send as a "message". This will require some playing around with how to parse the different record types, but all in all that looks like the cleanest design:
use exported variables from the calling script to distribute invariant variables to child processes (or positional arguments)
use a FIFO for everybody else to send specific information to any process.
Have just about completed shift over to this new methodology and CPU usage does indeed look significantly lower (not that CPU usage was ever a problem on my RT3200).
It only recently dawned on me that we could have one variable store FIFO to write values to variable store and then individual bridge FIFOs for each process for reading values from global store and sending to each process. It seems to work.
I do not think that trying to turn all/most variables into global variables is actually required... we really just seem to want to shave the cost down of getting per reflector information to maintain_pingers(), so I will go and just write them into maintain_pingers FIFO, problem solved (I want more generic parsing anyway that can differentiate between different commands, so shutdown requests can be sent the same way, and having one FIFO per process seems nicely symmetric...)
I'm only proposing this for those shared between processes. There are more pathways though like from the achieved rates monitor. So I think a variable store that is accessible from any process is helpful.
For some of the Lua autorate crowd, @lochnair, @_FailSafe, @CharlesJC and whoever else is interested, has anyone looked at the nim language https://nim-lang.org/ ? It compiles to C which should make it able to produce binaries for any OpenWrt platform and it seems like it's the right mix of high level language yet capable of doing the sort of stuff needed for nimble embedded purposes. Just a thought.
Hi all. First off thanks for all your work on this project. I don't know if this is the right place to ask questions about the project so let me know if it's not. I'm wondering if the issue with Starlink filtering out ICMP time response packets has been resolved yet. And is this something I could install onto an Edgerouter x. I am new to bluffer bloat and networking in general but looking for a way to increase latency stability on Starlink brought me to you guys. Thanks again.
Perhaps @gba can answer in respect of Starlink, but in respect of cake-autorate I'm personally still working with ordinary ICMPs via fping on my own 4G connection. This actually seems to work very well - I've had this running for months for day-to-day personal and busines suse. I understand that it works reasonably well for Starlink connections too and we even tried to compensate for Starlink satellite switching, but it's been a while since we have seen active testing of that.
You can try out cake-autorate on your Starlink connection using the code on the master branch of cake-autorate.
I have implemented support for timestamp ICMPs via tsping, but that is untested and probably needs some more work.
I have been very distracted by trying to make a variable store in bash using FIFOs as a replacement for temporary files for inter-process variables, but it is looking like this will not work out because bash uses byte-by-byte reads when reading from a named pipe, which kills performance.
So here is a question: we converged on using FIFOs for passing the delay records to the main loop, didn't we do so after cheching passing by standard file would be slower....
I am btw not convinced that reading byte by byte itself is a showstopper, after all to split the records read needs to do that anyway....
P.S.: Even if not for efficiency, FIFOs solve the concurrency/atomicity problem with multiple writers (if multiple pingers are used) in a pretty elegant way, so might still be a good enough solution for the delay records... also for the delay records we already multiplex a set of variables into each record...
Ah that is immensely helpful - I hadn't appreciated that and reminds me how much I depend on such insights. Does that mean that for streams with log lines and reflector data lines there would be no benefit in forcing fixed length reads, reading with "-N" and then rereading the read variable to split up into the individual records?
It seems that a huge benefit of reading from temporary files that I hadn't appreciated before is that lseek() is available. With a FIFO the data is consumed as it is read and any next character could be the new line to trigger a given read return, so reads are byte-by-byte.
But yes, FIFOs give benefits not so readily available with temporary files including those you mention and also the consumption of data on read, which is also helpful for our needs.
I'm hazy since that was so long ago now.
Yes it's definitely all good enough since what we've been doing works just fine on my RT3200 and can even work on @richb-hanover-priv's Archer C7.
Nevertheless I thought that compared to simple file writes and reads/rereads my var_store mechanism would be an improvement.
My idea was that individual processes write/read by sending write/read command to var_store FIFO, and a new var_store process reads from the var_store FIFO and sends write confirmation or read data back to each individual process FIFO, but this turned out to be very slow (circa max 50% CPU) compared to just writing/reading temp files (with rereading as necessary) for those variables (circa max 20% CPU).
I assumed that this byte-by-byte read would be the cause of this jump, but perhaps it's something else? Like needing to read at both ends for any read/write and separate process churning away in the background? I'd be really interested to know what you think.
(I want more generic parsing anyway that can differentiate between different commands, so shutdown requests can be sent the same way, and having one FIFO per process seems nicely symmetric...)