I think the work you guys are doing has applications beyond just augmenting SQM.
Having a way to characterize the latency profile of your internet service over time, with respect to time-of-day and bandwidth usage, would helpful for anyone ISP shopping.
It could also be great information to have when working with your ISP to resolve connection issues.
Well done!
And I'm screwing up my courage to post a bunch of questions. Maybe this weekend... Thanks.
@Lynx Update regarding stalls: the supposed definition (no pings, almost zero traffic) does not apply. Because of the newly added logging, now I am not blind into what happens between pings. The traffic is almost never zero, so it is "simply" a special case of bufferbloat. Please don't add any code based on the old definition to the master branch, as it's wrong.
What's common to all "stall" instances so far is:
I guess you should get a packetcapture to get to the bottom of this. How large are the added delays for the ping packets and how large is the load interval? So is there a smaller period of zero traffic followed by low traffic or is this mainly really low rate but no full drop out... (does the modem have anything that might be diagnostic here, like retransmission counters?)
About that ping time series that sure looks like all ping processing is stopped, pings* are queued up somewhere** and then released, like in our initial stall concept. Could you for testing try to ping though a VPN so your ISP can't specialcase ICMP packets?
*) That is either the request packets to the reply packets, this is something that OWDs would be able to tell us more about the direction.
**) Not in our own queue though, so either in the modem for upstream or in the basestation for downstream.
Do you think our existing treatment of stall based on no pings and very low load still makes sense? I mean it seems logical to me?
I think it still makes sense, although it was 'sexie when it also appeared to ameliorate @patrakov's odd link behavior....
I will try, but no guarantees. The complicating factor here is the need to keep the "fiber, with fallback to LTE" combo as the main connection, so I will have to write some mwan3 policies so that the VPN traffic always goes through the LTE link, and the pings and the test traffic generated by the laptop goes through the VPN.
Maybe just unplug the ONT for a test? That way mwan3 is forced to use the LTE fallback?
I would rather avoid this.
Fair enough. it is a rather disruptive test that does seem incompatible with normal network usage/
How about netifd PBR? Once set up you can just add entry in LuCi to route traffic to whichever destination. Works a treat albeit not so well documented.
I'd be curious if VPN use circumvents your ISP messing about with your traffic (assuming they do).
This would mean (because there can be only one PBR package active) dismantling my current mwan3 configuration - something that I wouldn't do without testing first on a separate router. The issue is not about routing the laptop through the VPN (that's easy), but about routing the IKEv2 VPN through LTE.
Why not WireGuard?
Commercial VPN provider.
Many offer WireGuard though don't they? Have you also considered VPS? It'd make two way measurement easy but I'm reluctant to make the switch since NordVPN has been great for me.
Please don't disrupt me, otherwise I will never start the test. Unfortunately this VPN provider has the WireGuard option only available in their client app, which is already too fishy.
NordVPN are the same but you can use their app to ascertain the WireGuard credentials and use those. Maybe same applies for you too.
Happy testing!
Done. Actually I hit a bug in the xfrm interfaces (they drop 50% of NATed packets), so had to switch to strongswan-mod-kernel-libipsec, which works.
I will set up a packet capture and OWD ping train later today, but I cannot exclude the possibility that the ISP deprioritizes IKEv2 packets.