High bitrate streaming - jitter and latency optimising hardware choice

Apologies for length of post, I'm not sure how to condense it without losing relvant information

From two weeks ago not kowing anything about linux kernels, packages, soc's, developemet boards and router internal componentry etc my eyes have really been opened thanks to everyone on these forums for which I am super grateful.

Im tyring to understand how best to design an openwrt solution.for my needs and what to priotritse to minimise jitter and latency.(I'm focussed on internal Lan performance for my use of high bitrate high framerate game streaming from my PC to my Quest 3 VR headset. My ISP internet is only 137/20 so WAN optimised is not a concern.

Current system & use

PC RTX 4090(2.5GbE capable) > Asus AX58u(merlinwrt) > Quest3(5Ghz/160/DFS)

When streaming fast paced simracing I always experience the odd microstutter in game rendering on Quest3. Testing in DFS as that is the cleanest environment.in my neighbourhood. Non of my neighbours know to use DFS and its only on beginning of weekend that I've noticed it jump out of DFS channels sometimes.

Testing and elimination

Through testing I have (most probably, I dont have knowledge or means to conclusively) determined that:

  • PC system (7800X3D & RTX 4090) is more than capble of stutter free frame generation at even higher Hz than Quest3 120Hz

  • GPU to Quest3 video stream encode/decode throughput/sync issues - when using meta link USBC cable the stutter is virtually eliminated at even higher bitrates, suggestign enc/dec chain is not cause of issue.

  • Wfi 5GHz band is empty at DFS channels and stable, regular 5GHz channels too much interference from neighbours. No other wireless devices connected to AX58u. So can sustain over 900mbps throughput up and down consistently on wifi. Quest 3 bandwidth utilised during streaming around 500mbps

Which leads me to suspect the issue could be in the underlying network hardware. I can acheive full sustained bandwidth utilisation (900+ mbps up/down) when testing from PC to Quest3. But full bandwidth test doesnt illustrate the lags and spikes that affect VR streaming.

Solution

So have embarked on a mission to build a router and wifi soution to attempt to address this microstutter situation in network hardware.

Am I likely to see noticeable improvement in this regard by moving from the Asus AX58u (BCM6750 + BCM4384) system to an openwrt build on:

  1. GL.iNet MT6000 (MT7986 + MT7976, HWO+WED)
  2. BPI-R3 (MT7986 + MT7975, HWO+WED)
  3. x86-64 (i5-7500 + RTL8125quad nic + MT7916 wifi AP

Your router is based on broadcom SoC that is not well supported by opensource drivers as per https://github.com/RMerl/asuswrt-merlin.ng/blob/a260ae3a49b788cfc513b780bacc8e6301f00553/release/src-rt-5.04axhnd.675x/kernel/dts/6756/RT_AX58U_V2.dts
i.e OpenWRT if ever supported will not make your wifi better than OEM firmware

You can try to optimize merlin QoS and ACS parameters measuring down the road:

While choosing from OpenWRT capable gear:
https://openwrt.org/toh/views/toh_available_16128_ax-wifi

Thanks for the feedback brada4, I'm afraid you misunderstood my question. There was no mention of putting openwrt onto the AX58u in the hopes of improving wifi from OEM/merlin firmware.

.I know the post is a lot to read so I apprecaite you taking the time to go through it. I'm not able to conclusively identify where the issue arises from hence the brealdown of all the 'testing' I have done to try eliminate/identify source of the probllem and the explanation of what i am seeking as a solution by migrating to openwrt on a completely different hardware platform.

I have previosly used the waveform bufferbloat test but I understood that test is for bufferbloat over WAN. And my AX58u does perform poorly there. Would that result be indicative of degaded throughput performance between two devices on the Lan that are pushing a lot of packets between themselves (high res/fps game stream from PC to VR headset)?

I'v foud this which seems to indicate so

But with regards to using QoS to reduce bufferbloat, if the ONLY traffic on the router is the (local) game streaming from PC to VR headset, there are no other packets to lower the priority of versus the packets I want prioritised so will Qos make any difference? (No other devices conncted to this router only PC and Quest3)

Likely occulus uses TDLS, ie STA-STA interconnection outside AP(cf chromecast) , and the AP QoS or its trivial version of priority plays no role whatsoever here.
You need to dump wifi traffic shortly to confirm packets flying directly between PC and display. Airtime fairness MAY do what it is supposed to do leaving fair time for your streaming. There should be no problem feeding 4k @ 120Hz taking 15...20 Mbps (bits, not octets i.e 2 headsets can stream in 2.4ghz)

Any filogic CPU will forward 2.5Gbps without wed offload. Qoalcomm equivalent will need proprietary offload drivers, which one day may not work with linux kernel anymore.
For the expense I'd go for 3-radio Wifi 6E to add extra year in service.

1 Like

I've not encountered the terms TDLS and STA before, Im only two weeks into discovering and learning about this entire field of routers,wifi radios, linux and openwrt with its assocaited hardware, so this a great launching point for me to do wider research and learning to broaden my understanding, thanks @brada4


There should be no problem feeding 4k @ 120Hz taking 15...20 Mbps

A lot of PCVR users tend to push much higher bitrates than that anything from 200-900mbps at higher than 4k render rsesolutions.


Re: the wifi 6E radio, while lurking I read a few posts here discussing deployment of the asiarf aw7916 module and am considering to pair that for 6Ghz capability with either the x86-64 platform or some filogic 830 (potentially BPI R3 as already contains 2x2.5G SFP and 1x mini pcie for add in module)

You mention that any filogic cpu will forawrd 2.5Gbps without wed offload. Where can I uncover more information like this please. I've discovered sites like techinfodepot and wikidevi to delve into the internal details of hardware but that only gets me so far.

Compared to filogic cpu's, I'm led to believe x86-64 is more performant in this context even against the beneficial inbuilt Mediatek hardware accelarations albeit at a considerably higer power draw?

My interst in the x86-64 platform over filogic is simply due to the easy modualrity of starting with an sffpc base with 2 pcie expansion slots for various classes of nics and ap's as my use case or need for experimenting grows.


dump wifi traffic shortly to confirm packets

That would be where I figure out how to use wireshark right?


I'm really appreciating your patience and knowledge, its helping me a iot

TDLS: it is an extension to AP - client ( called station or sta) architecture for those clients to communicate directly IN SAME CHANNEL.

It is just 3k @ 90Hz

Speeds: count 10x memory copy as simplified "forwarding" speed estimate.

Wireshark: more like aircrack-ng, but yes.

1 Like

That source of latency has already been eliminated as the PC has always been conected via ethernet (currently 1GbE as the AX58u does not have 2.5GbE ports and there is no other traffic on the network when I am using the Quest 3, this is a dedicated wifi router specifically for the Quest 3.

There is a wired access point from the AX58u which serves the lower floors but if I am upstairs using the Quest3 then there is no device which needs traffic to be routed through to that AP other than a printer. That lower floor AP operates on 2.4Ghz and a distant 5Ghz channel nowhere near the DFS channel used for the Quest3 on the AX58u.

Which is why I previously asked how could QoS/SQM provide any further imporvement if there are no other packets on the network to give priority over. (brada4 has answered that with - "Likely occulus uses TDLS, ie STA-STA interconnection outside AP(cf chromecast) , and the AP QoS or its trivial version of priority plays no role whatsoever here.")

I've eliminated external factors as cause of the microstutter as much as possible and from the varius other testing have possibly identified the underlying network hardware/drivers/frmware as the weak point hence the question regarding different platforms with openwrt.

There is no guarantee 400Mbps encoder on your PC keeps stream rate that at no point exceeds receiver 400Mbps leading to decoder in headset legitimately stutter. Nothing to do whether it is wifi or cable connected, decoder will choke even at minimal bitrate fluctuations.

To try and eliminate if decoder is choking or not, instead of streaming over network at 500mbps bitrate I tested streaming via USBC cable to headest and at 720mbps bitrare it is consistently better and microstutter free.

Would that not be indicative that the issue was not in the decoder, if it can perform well at even higher bitrates when not using the network to transport the video stream?

No relation with cable, your wifi bandwidth is more than sufficient, set your stream to defaults so that devices work within specs.

Thank you so much for affirming my suspicions.
As much as I try to learn and think I know something, I am sensible enough to accept that I dont know enough and that there can always be other factors which have a greater impact than I realise. Which is why I'm always happy to defer to those who have more experince and knowledge.

I only just realised (in basic terms) what switches really do in terms of forwarding packets via MAC address. Previously I'd only ever casually thought of them as hubs without any packet handling ability, so have ordered myself a multigig switch (RTL8373) with alleged backplane throughput 60Gbps and packet forwarding rate 45Mpps. But I suspect there is a reason it was so cheap compared to more recognised branded switches. If anything it has to be somewhat of an improvement over the much older Broadcom BC6750 in my current Asus AX58u all in one device.

Thanks for all the pointers. I'm planning to spend today faffing about with openwrt on a modified dual rj45 laptop just to get familiarised before I get my ISP switched over and order the x86-64 box for openwrt (and nextcloud) duties

id recommend running wireshark while streaming to eg drops and retransmits due to giggles not having buffer for fast tcp.