I don't understand something fundamental about this SQM business...how is limiting your connection supposed to improve overall quality??
After following the advice given to me in several other threads (won't link to them here since they're long and boring ), I have just setup SQM on my TP-Link Archer C7.
DSLReports test before SQM, without optimising the report settings as suggested here:
DSLReports test after optimisation:
DSLReports test with optimised settings and SQM:
Can anyone please explain these results to a dummy like me, and also answer my initial question? MTIA!
P.S. Apologies for the link images not working; I'm not sure how to facilitate that in Markdown. If anyone knows how to fix them, please PM me. And the first test was from before I signed up, and having chosen "Cable" instead of "DSL". Not sure if that makes a big difference.
Yes I linked to your thread about settings optimisation in my OP...that's what I meant when saying "before/after optimisation" The suggestion you make about Linked BBCode is what I was trying to acheive, but in Markdown, since the BBCode didn't seem to work either.
LOL no worries; I wasn't even expecting that much of a reply after so little time, so thanks mate
Quality in SQM context means latency, ping, responsiveness, or however you want to describe it.
"Before" you had clogged down your bandwidth with too much traffic, which got queued into buffers, making the roundtrip to websites quite slow, although the measured total traffic per longer period was high. Latency was 100-900 ms, which would have made e.g. gaming impossible.
"After", with SQM, packets are selectively prioritised, some queued, some even dropped (for apps to re-transmit them). That decreases the total traffic volume a bit, but makes the roundtrip much shorter. Your latency dropped to 10-20 ms, which is quite ok and internet feels responsive.
Apologies mate; I admit I probably should have researched this a lot better on my own before whining to you fine folks.
And what you said about the differences totally makes sense; thanks for breaking it down. I'll take the advice at the bottom of the guide about tuning things further, and see how that improves my speed rating.
But even if it doesn't improve, I guess you're saying that my internet responsiveness should be greatly improved. So after a few weeks of monitoring things closely, I'll report back on whether you're right
Easy test. Turn sqm off, Set a continuous ping running to a major ip/service (google.com) then download a huge file, like a Linux iso image or something else massive that will max out your bandwidth. While that’s downloading try and browse the web for a few websites or pages you’ve never been to before.
Now turn sqm back on and repeat the test. If tuned correctly for the bandwidth you should see a huge difference in responsiveness of the browsing. And the ping times should have barely increased ,+10 to 20ms above your normal ping time is typical with sqm enabled..
Thanks, that's very good to know and I'll try it tomorrow as I'm taking a break from all this technical stuff right now. I literally have a headache after spending the entire Easter weekend fiddling with my router and USB HDD.
I've slept about 15 hours out of the past 80 and taken a few short breaks to eat, but other than that I've been constantly overloading my brain with new information LOL.
SQM is like an alien Star Trek spaceship. Imagine you're driving down a freeway, and you hit one of those irritating spots on the freeway where there was an accident 4 hours ago, and because of that, there's a 2 mile backlog of cars. The traffic is free and clear AFTER the previous site of the accident, but leading up to that spot, it's going super slow, and you have to sit in this line of cars all moving at 2 miles per hour. This is like a buffer backlog.
Now imagine an emergency vehicle needs to get through. It needs to get to the other end of this backlog FAST or it will be too late to do any good.
There are two ways we could do this:
Prevent the backlog from happening in the first place... by beaming backlogged cars back to their origin using the star trek transporter... Without a backlog, it's easier to maneuver and packets don't have to wait in a long line.
Keep an "extra lane" open for emergency vehicles... Only they can use it, and they fly through without delay.
SQM is like both of those strategies put together. It drops packets, which is like an alien spaceship that comes along and just beams cars back to their house. It looks at this enormous backlog of cars, 2 miles long, and maybe it just beams 2 out of 3 cars back to their original starting point... Suddenly there's room to maneuver on the freeway, everyone can put their foot down on the accelerator, and within seconds everyone's moving at full speed... Everyone who's been beamed back to their starting point just has to set out again and start driving, but they're not congested either, so they go at high speed...
This is the basic theory, but then in addition to that it's possible to use something like diffserv4 which is like extra lanes... If your packet has a "flashing light" or an HOV lane sticker (ie. a DSCP tag) it can move into one of the reserved lanes, and skip the wait.
That's almost literally how it works. It keeps all the packets flowing by dropping packets which is like sending them back to their starting point, and keeping lanes open for high priority stuff.
LOL love the Star Trek analogy. The article that hnyman linked to was also a very amusing and interesting read. And both road-traffic-based analogies break the problem and the solution down very well.
OK so I now know why SQM is needed, and how it helps. Now I just need to do some testing like Sparks suggested and figure out the right settings to use for my connection.
To that end, does anyone know of a site (or better yet, several) that I could constantly hit again and again without any caching or AJAX requests producing false results? I've ran a quick test on dailymail.co.uk and it worked the first time (responsiveness with SQM off was horrid, but slightly better with SQM), but I think I need to do a lot more tweaking and their server caching will get in the way.
reuters.com seemed like another good test candidate, until I realised that their home page was still loading AJAX crap after 5 minutes LOL. So perhaps I should just stick to DSLReports tests.
OK after a bunch more tests at dslreports.com, I can see why Sparks suggested testing this in the way that s/he did. The effect that SQM has on actual speed is easily noticeable by doing those tests, but makes things seem worse than they probably are.
As hnyman, dlakelan and https://www.bufferbloat.net/projects/bloat/wiki/Introduction/ made me realise, the benefit of SQM is in perceived speed, notactual speed. Websites should be (and are) seemingly-loading faster, even though your link speed has been greatly reduced (by 10 - 20 mbps in my case!).
This makes testing for the "sweet spot" in SQM settings incredibly difficult, or at least harder than I imagined. But for now, web performance does seem better with SQM enabled, so I guess I shouldn't care until I start noticing some serious lag.
Thanks again everyone; as I've come to expect you're all incredibly-gifted and willingly-helpful individuals and I hope to be of as much value to the community one day
Speed is really two things, bandwidth, and latency. SQM reduced bandwidth slightly and this causes a major reduction in latency. Usually you only need to drop bandwidth by say 5% from its tested maximum and your latency will drop from seconds to milliseconds
Well, SQM trades off bandwidth versus latency-under-load, so you simply try different shaper bandwidth and look at the bufferbloat detail plots of the dslreports test, if you shape too high (i.e. if you do not really shape at all) the bufferbloat will increase/jump up noticeably, once you reach roughly the sustainable bandwidth bufferbloat will reduce to an initial spike for the download direction and no spike for the upload direction.
You will need to to decide how much bandwidth you are willing to trade-in for lower ;atency-under-load; a trade-off that becomes less tricky to strike once you realize, that all interactive traffic, like VoIP, video-conferencing, on-line gaming, and with-in reason even simple web browsing is more latency- than bandwidth-limited (that is once you cross a bandwidth threshold).
Final note, download direction shaping is a bit approximate so it generally is more robust/reliable to set the shaper a few %age points lower than the threshold rate of the actual bottleneck link.
Yeah that's the exact formula I used to obtain the value entered into LuCI -> Network -> SQM QoS -> Basic Settings -> Download speed, as per the recommendation in the official wiki guide.
What you just made me realise though, is that my neighbourhood traffic has decreased significantly since then. It's almost 2 AM now and I set this up at 7 PM last night, i.e. I chose the worst possible time to start testing and haven't changed the download speed setting since!
I'll fix that and LYK if things change (which they almost certainly will!)
Actually if you want SQM to work well at all time you need to set it to a few percent below the MINIMUM speed you get, otherwise things will become high latency when everyone in your neighborhood starts streaming
Yes, initially it is a good idea to repeat the speedtests/bufferbloat measurements a few times per day (at different expected network load situations) that way you can iteratively hone in on a relative robust and reliable setting for the shaper (or you might learn that the spread over a day is too large and you need multiple settings and change between them based on the time of day).
Hmm yeah that makes sense I guess...but I'm pretty happy with my most-recent test results and the current browsing experience.
If/when I need to and/or can be bothered, I'll download DSL Reports' cmd tool and run some streaming tests at the same time (as I'm sure you're aware, the browser-based test requires not switching tabs while the test is running).
I think this is very likely to end up being the case, as I've observed it in the past. Right around 7PM, bandwidth plummets and latency skyrockets, which is to be expected I suppose (classic definition of "peak hour"; ISPs that do bandwidth shaping generally start it at this time).
Around this time of day, things are generally good and that works well for me since I've always been a night owl. Mid-morning - early afternoon performance (around 7AM to 1PM, when I've rarely been awake at that time LOL) has been better than peak hour but still much worse than early morning (1AM - 7AM). And mid-afternoon (1PM - 7PM) is generally the same as mid-morning.
So how would I automatically accomodate for this? Can I setup four cron tasks to run at the appropriate times, each one changing the shaper values? If so, how?
EDIT: The answer to the above question, if I'm not mistaken, is to use sed or something similar to modify the lines option download 'x' and option upload 'y' in /etc/config/sqm, replacing x and y with the appropriate values. I'll research exactly how to do that with cron if/when I need to.
the star trek analogy blew coffee through my nose. thank you.
there's lots of other analogies, van jacobson uses a fountain one, stephen hemminger uses water bottles, I use people as packets. I had a fun talk here recently:
but I'd love incorporating a star trek theme as the transporter analogy really makes sense. You'll also find so many other present-day post-covid car transit studies, where traffic was down 60% and speeds up 4x, where the same queue theories apply.
), I have just setup SQM on my TP-Link Archer C7.
The suggestion you make about Linked BBCode is what I was trying to acheive, but in Markdown, since the BBCode didn't seem to work either.
