Thanks Bill! Perhaps that was the case. Frustratingly I was encouraged by the TalkTalk provider to upgrade because it was a free upgrade but wasn't made clear that the upload speed would be sacrificed so dramatically in order to achieve a 20-30Mbps increase in download speed. Feeling like I've gotten a bit of a rough end of the deal now.
I'll test SQM throughout the day and see if there are any performance improvements.
Mikrotik, is an unsupported OS better than the stock OS?
Both are linux based operating systems, RouterOS stock configuration is Eth1 is the WAN port, OpenWRT Eth1 is the WAN Port.
Mikrotik has a steep learning curve, the RB750gr3 or hEX mentioned has the same chipset as the Xiaomi R4A Gigabit, but a couple of extra ethernet ports, twice the RAM, and a USB port and microSD Card slot.
RouterOS7 and OpenWRT are pretty equal in performance at present.
If you are uploading that much, then the your current upstream speed is a downgrade in the quality of your connection and it may be worth forcing TalkTalk to regrade you to FTTC/VDSL
BT Openreach doesn't allow isps to sell G.fast unless the expected speed is 120Mbps or greater (not sure if that is actual throughput or sync rate), so the line must have been pretty marginal, or may have failed to reach its targets
The Xiaomi R4AG arrived today and I've replaced the HH5a with it. So my current setup is G.Fast MT992 ←→ Xiaomi R4AG ←→ BT WiFi discs.
Although there's some improvement, something still isn't right even with this faster router. Here's what I've observed:
WiFi or Ethernet to the R4AG via MacBook gets decent speeds.
12ms ping, 105Mbps Down, 8Mbps Up
With SQM enabled:
12ms ping, 80-90Mbps Down, 6-7Mbps Up
When my Desktop computer is connected via WiFi (I don't have s long enough cable to run an ethernet directly to the router) and running GDrive and cloud backup everything on the network grinds to a halt. High ping latency and slow up and download rates.
Ping 300-400ms, 20-40Mbps Down, 0.5-2Mbps Up
Enabling SQM helps to keep some of the network usable but the WAN performance is still taking a hit.
Ping 100-200ms, 60Mbps Down, 1.5-2Mbps Up
As soon as the desktop computer is turned off or the Cloud and GDrive services are turned off, then the network WAN performance improves to expected levels. I have a deficit of about 5gb of unsynced data on my Desktop that needs to sync to GDrive and 2GB to sync to cloud backup. It appears that GDrive is trying to sync 10-20 files at one go, so it could be the number of concurrent connections causing an issue?
Adding a rate limit in the GDrive app of 50-200kBps appears to help for a short period of time but soon the WAN performance will drop off a cliff.
It appears, once constant and sustained uploading of data is initiated, the network WAN performance takes a massive hit to latency, download bad upload that affects every client.
Various articles regarding TalkTalk 150 speeds indicate Download Speed (average): 147Mbps and Upload Speed (average): 29Mbps. My downloads speeds aren’t far off but the upload speed seems to indicate something really isn’t right with this connection.
Prior to upgrading from TalkTalk 65 to 150, I have stats daily (Jan-Feb 2022) stats from a SpeedTest that’s run on a schedule on a HomeAssistant instance. Every day I had a solid connection of 10-20ms ping, 75-80Mbps Down and 16-18Mbps Up. The upload speed is on a par with the advertised average speed and download is better than average when I’ve looked at the speeds in noted online.
Is it possible that I've configured something wrong on the router? I’ve mimicked the same setup that I had with the HH5a but from a fresh install. There are no configuration options I can find for the G.Fast MT992. Could the 320m from the cabinet and a higher than expected SNR (open reach engineer mentioned this when they last visited) be causing the issue?
The only thing I've not been able to test yet is connected the window desktop directly to the G.Fast modem. I'll try and find a long enough LAN cable in the next few days as the two are far away from one another.
Based upon the speed test performed when connected directly to the G.Fast modem, it sounds like I'm not getting to that 120Mbps down margin threshold. Are you indiciating that G.Fast probably isn't going to work in my home given the current Openreach hardware in the street and my distance from it?
fwiw, if it was me, I'd complain to TT and say you didn't expect the upload to be less than half what you had previously, and demand to be returned to using VDSL. That's assuming the new VDSL connection will be as good as what you had previously.
Is 320m distance to cabinet measured direct, or by following likely paths the old telephone wires take around the roads?
Imho, unless it is less than 200m, it may be hit or miss with regards to benefits of Gfast.
From an SQM perspective, if your uploads kill the network performance not only for the uploading computer, but for all machines in your LAN, try the following:
The two tricks here are a) the dual-xxxhost keywords which instruct cake to share capacity fairly between internal IP addresses, that is your GDrive uploading computer will not be able to hog all/most upload capacity if other machines want to send something (but without competition the GDrive upload should still saturate your uplink). And ingress instructs cake to essentially automatically adjust the download shapers aggressiveness to the actual responsiveness of the incoming traffic. See https://openwrt.org/docs/guide-user/network/traffic-shaping/sqm-details for more information (also how to do a quick and dirty test whether the router's CPU might be overloaded).
Note with these shaper-rate settings you can at best expect the following TCP/IPv4 goodput (what speedtests measure and report):
9.4500 * ((1500-20-20)/(1500+44)) = 8.94 = Mbps
7.110 * ((1500-20-20)/(1500+44)) = 6.72 Mbps
as shaper rates are gross rates, while speedtests report net payload throughput...
I would, unless the router is already overloaded, set the shaper to 105000 and 8000 respectively, so those speeds that you get without SQM as speedtest results (and only decrease these if latency under load suffers too much).
Are you testing the ping from the same machine you are downloading from? Do you have a dedicated AP or does your router need to do both sqm and WiFi?
And do perform the MTR tests, Bill recommended... you can BTW run mtr from your router:
use opkg update ; opkg install mtr if mtr is not already installed and invoke it like: mtr -ezb4w -c 120 pingbox1.thinkbroadband.com
for getting a nice report for ~2 minutes of pings (you can copy and paste the terminal output here in the forum), or use: mtr -ezb4 pingbox1.thinkbroadband.com
to get a continuous updated mtr output in the terminal.
Side-note: pingbox1.thinkbroadband.com seems to be the host that Thinkbroadband uses for their network quality monitoring, so running mtr against that allows you to compare your results with those recorded and displayed with a thinkbraodbandprobe giving you a view from both sides.
I completely forgot about the tools you’d suggested Bill. I’ve setup a Broadband Quality Monitor test on thinkbroadband, this morning. I’ve only had the test running for an hour so not much to report in the below image. And my network / desktop computer isn’t on yet so there’s not much traffic.
I'm going off the conversation with the Openreach engineer when they visited the property. I believe their testing equipment measured the line from the cabinet to where it runs into the block of flats at 320m. I live on the 2nd floor of a tenenment building so there will another telephone line that brings that into the flat property and to the final master phone point where the router is connected. It's my understanding the Openreach engineer wasn't able to measure the distance from the common area entry point to the master socket in my flat.
I found SCAN computers were selling the R4AG both on their own site and eBay. With next day delivery.
I've tried both, I've run a ping test on the desktop uploading to Gdrive. I've also tried the ping whilst the desktop is on, but running it from my phone and on an ssh connection from the router. WiFi is using a separate BT Whole Home WiFi AP mesh network. The master WiFi disc is connected to the router via a LAN cable.
Thanks for the SQM and mtr suggestions, moeller0. I'll see how they stack up this morning.
Here's the output of the MTR tests which I ran from an ssh on the R4AG router. I'd implimented the suggested SQM changes from moeller0 whilst this test was running and my desktop / GDrive machine was off:
Nothing jumping out, the "Wrst" column is a bit higher than I would expect, but since it is mostly the end-host that gives reliable reports, 12.3 to 21.6 is not great, but also not terrible, also no packet loss (but then this were just 120 packets).
For getting a better handle on loss, maybe let mtr run continously while you are away: mtr -ezb4 pingbox1.thinkbroadband.com
and just look at the results after a few hours (and copy and paste here)....
Also interesting to repeat this mtr-experiment while you are actually loading your link.
And finally, would e good to see the output of tc -s qdisc...
Agreed, I think it's poor upload speed from the property to the cabinet or somewhere else outside the property that's causing the issue. A direct connection the G.Fast modem never peaks beyond 8Mbps up. I'll try and get TalkTalk to switch me back to TalkTalk 60 / VDSL.
For future reference:
The latest SQM settings have done a good job on the R4GA.
The ping latency stays around 10-20ms even with the network under load (desktop computer on with GDrive running and uploading multiple files).
The download speed stays establish, usually around 90Mbps down but sometimes drops to 60-70Mbps. Thankfully the new SQM settings aren't causing the download rate to drop off a cliff when files are being uploaded. With the desktop computer uploading the upload rate is pretty poor though 1-2Mbps.
I've left the desktop uploading most of the morning but was unable to leave a local instance of mtr running. However, the Think Broadband test results are below collected from the on-going test on their side.
That computer is connected via WiFi? Could you, just for a test, run a wire between OpenWrt router and the desktop computer to figure out whether the issue might be related to what happens on the WiFi link?
Correct, it's connected via WiFi. I've ordered a cable which should arrive in the next couple of days to try this test. I currently have one that will stretch from the Desktop to the modem. Seems like the last logical thing to try.