CSGO vs CS2 using SQM

Xsanchi · September 11, 2023, 1:17pm

I am a huge fan of Counter-Strike and have been playing CSGO for a couple of years using OpenWrt with SQM. Never had an issue while playing online even when it comes to all my bandwidth being used.
The SQM does the job perfectly even though my internet speed is not that great at all, 20 MB download and 1 MB upload speed "which is the most used in my place".
Now CSGO soon will be replaced by CS2, The new game is still in beta and I have got access to it recently. played a few games but it was unplayable at all. so much lag and very high packet loss all the time! talking here when other family members use the internet.
have tried a lot of changes on sqm but nothing seems to help.
I made a video that explains everything and I am wondering if this issue is from the SQM not being able to do its job with this game or it's from the game itself since CSGO is running on tick rate servers 64 or 128 in other platforms like Faceit. but the new CS2 is using a new SUBTICK system instead!
anyway here is the video I made which shows a huge difference when I play CSGO compared to CS2.

moeller0 · September 11, 2023, 1:52pm

You should try to get some packet captures for both games (play for a few minutes and write down your subjective experience and/or record a video capture of the game).

Also save the output of:
tc -s qdisc and tc -d qdisc from just before and just after each game.

ATM, I ave no idea, it might be that the new game simply not intended for 20/1 type of link speeds...

BTW, what is the output of cat /etc/config/sqm ?

Whether this is fixable is a different question, but first we need to understand what is exactly happening that makes the CS2 so much less resilient... (Side-note if you use sqm scripts with per-internal IP fairness, then running the up/download stressor on your gaming computer will effectively evade the per-internal IP fairness as this is based on your internal IP address). But really packet captures might be helpful, maybe start with a bit of game play (say 1 minute for each game) with no other network load.

Hudra · September 11, 2023, 6:21pm

Please have a look at this Topic:

I don’t have any experience with CS2 but after a quick search I found this video about the the subtick system you mentioned. What I can see is that the packet size drastically increased with CS2’s Sub-tick servers + it seems like there are now 2 packets sent per tick (or could be fragmentation):

So it looks like the game uses way more bandwidth now and with your limited upload bandwidth this could be the reason for your issues.

moeller0 · September 11, 2023, 6:57pm

At 1 Mbps upload a single packet of 1301 bytes takes roughly

1000*(1301*8)/(1000^2) = 10.408 Milliseconds
((1301)*8)*64/1000 = 666.112 Kbps
and if we add both packets we get roughly
1000*((1301+300)*8)/(1000^2) = 12.808 Milliseconds
((1301+300)*8)*64/1000 = 819.712 Kbps

That is essentially 82% of the gross upload capacity, quite a lot I would say....

But that also gives a [possible way for sqm to misfire, if there are other active users.... that is because with even only a second active upload user the gamer's capacity share will be on average only 50%, but at least in this subtick example we are well above 66%...

But maybe the OP could get some packet captures to look at, so we do not need to extrapolate from youtube video's of other gamers?

Xsanchi · September 11, 2023, 7:20pm

thanks for your replies and the attention guys
I was looking around at how to do the capture packets.
not sure if it is meant to be done with Winshrake!
but then I read some articles that this could lead the one to vac-ban by steam.
I still can test this on a new account with csgo but the new cs2 is still in beta and to get access to it, must have a prime account, etc.
so I don't know what must be done now but I have contacted the counter-strike developer via their email which is provided on their Twitter page for reporting any bugs.
about the game requires more than 1 MB upload to run fine.
as you may see in my video when I stopped the stressor that I was using, the game was really playable with stable ping and no packet loss at all.

moeller0 · September 11, 2023, 7:32pm

That sounds rather unlikely, how would steam even know that you capture packets on your OpenWrt router? The only catch is that packet captures get large quickly, so you should only do them on a router with sufficient storage (e.g. on a router with an USB port, plug in a USB stick and save the capture onto that stick.)

Xsanchi · September 11, 2023, 7:37pm

@moeller0
I really did not know that I am able to do that via openwrt. if this is what you mean exactly. then I will see how this can be done and will share what you have asked me to provide asap.

moeller0 · September 11, 2023, 8:14pm

First install tcpdump on your router:
opkg update; opkg install tcpdump
Then add a USB stick and mount it e.g. to /SRV
Then, assuming your wan interface is pppoe-wan do

tcpdump -s 100 -i pppoe-wan -w /SRV/my_cature_file.pcap

-s 100 captues only the first 100 bytes of each packet (so the headers)
-i pppoe-wan captures from the pppoe-wan interface, replace with the interface you are interested in
-w /SRV/my_cature_file.pcap will save the packet capture file to a file with name my_cature_file.pcap in the /SRV directory (make sure to select a directory and name to your liking)

After getting a capture, use scp/sftp/winscp to copy the file off the router, you can then load it with wireshark and look inside.

Typically for me the first capture tends to be a bit big, but can still be a good starting point; later one can restrict tcpdump to only store a sub set of the packets... but let's start quick and dirty.

BTW there is also this wiki page:
https://openwrt.org/docs/guide-user/firewall/misc/tcpdump_wireshark#:~:text=running%2022.03.2-,Capturing%20packets%20from%20an%20OpenWrt%20appliance,be%20processed%20by%20other%20tools.

Lynx · September 12, 2023, 10:07pm

Can you expand a little on the calculations? What do the different lines represent?

moeller0 · September 13, 2023, 6:48am

Sure

1000*(1301*8)/(1000^2) = 10.408 Milliseconds

To calculate the transmission time for a single packet we take
1000 [msec/sec] * (packet_size [Bit] * 8 [Bit/Byte]) / (1000^2 [Bit/sec])
that is we convert the packet size (estimated from @Hudra's Sub-Tick table above) from Byte to Bit and divide by the approximate rate (the OP mentioned 1 Mbps upload rate in his first post) to get the transmission time in seconds, finally we multiply with 1000 to get the time in milliseconds.

((1301)*8)*64/1000 = 666.112 Kbps

Here we calculate the rate required for the packet we calculated the transmission time above , so:
(packet_size [Bit] * 8 [Bit/Byte]) * packet_rate [Hz] / 1000 [Bit/kBit]

The table tells us that there are 64 packet pairs per second so we simply add the approximate sizes of both packets and pretend we would send one larger packet instead of 2 packets of different size to simplify the calculations a bit we end up with:
1000*((1301+300)*8)/(1000^2) = 12.808 Milliseconds
((1301+300)*8)*64/1000 = 819.712 Kbps
for the whole sub-tick caboodle.

Now, I might have made a mistake somewhere (would not the first or last time) but the above describes what I intended to calculate. The point is if these sub-tick packet sizes are correct then CS2 might simply to bandwidth intensive for a shared 1 Mbps upload. One potential work around is is to create a dedicated qos-script that allots the uplink capacity unfairly, giving the game enough capacity... However the OP first should confirm that:
a) the upload truely is as problematic
b) the issue also persists with uploads from other machines in his network after configuring cake for per-internal-IP-fairness.

However one issue ihere s ACK rate, "normal" TCP Reno will send one ACK for every two full MSS segments received (and/or a timeout value), which assuming MTU ~1500 and normal ACK sizes ands up at ~1/40 of the reverse data rate. at 20Mbps we get 20/40 = 0.5 Mbps upload required for reverse ACK rate... so even pure (TCP) download traffic by other machines is going to eat significantly into the OP's upload capacity and might partially cause the issues he sees. Here maybe cake's ACK-filter might be of help.

@Xsanchi could you please post the output of the following commands run on your router (ssh into the router, run these commands on the command line and copy and paste the output from the terminal window here as "Preformatted text"*):

cat /etc/config/sqm
tc -d qdisc
tc -s qdisc
ifstatus wan | grep -e \"device\"

that should give us an idea about your current sqm configuration.

*) Simply sandwich the pasted text between two lines only containing 3 backticks each, in the editor this looks like
```
Your text here
0
00
000
0000
```

which will be rendered like this:

Your text here
   0
  00
 000
0000

Note how the fixed width font of the "Preformatted text" rendering will show the numbers well-aligned?

Xsanchi · September 16, 2023, 4:25am

Hi again
and sorry for the delay but I was waiting for a friend who has some knowledge about things you have asked me for.
firstly let's start with the outputs

cat /etc/config/sqm

config queue 'eth1'
        option interface 'pppoe-wan'
        option debug_logging '0'
        option verbosity '5'
        option enabled '1'
        option qdisc 'cake'
        option qdisc_advanced '1'
        option squash_dscp '1'
        option squash_ingress '1'
        option ingress_ecn 'ECN'
        option qdisc_really_really_advanced '1'
        option iqdisc_opts 'nat dual-dsthost ingress'
        option eqdisc_opts 'nat dual-srchost'
        option script 'simple.qos'
        option download '17000'
        option linklayer 'atm'
        option overhead '44'
        option upload '950'
        option egress_ecn 'ECN'







tc -d qdisc

qdisc noqueue 0: dev lo root refcnt 2
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum 1514 ta                                                                                                                                                             rget 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc noqueue 0: dev br-lan root refcnt 2
qdisc noqueue 0: dev eth0.1 root refcnt 2
qdisc noqueue 0: dev eth0.2 root refcnt 2
qdisc cake 801d: dev pppoe-wan root refcnt 2 bandwidth 950Kbit diffserv3 dual-sr                                                                                                                                                             chost nat nowash no-ack-filter split-gso rtt 100ms atm overhead 44
qdisc ingress ffff: dev pppoe-wan parent ffff:fff1 ----------------
qdisc cake 801e: dev ifb4pppoe-wan root refcnt 2 bandwidth 17Mbit besteffort dua                                                                                                                                                             l-dsthost nat nowash ingress no-ack-filter split-gso rtt 100ms atm overhead 44






tc -s qdisc

qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 52665430584 bytes 68561358 pkt (dropped 0, overlimits 0 requeues 603)
 backlog 0b 0p requeues 603
  maxpacket 1514 drop_overlimit 0 new_flow_count 592 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br-lan root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev eth0.1 root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev eth0.2 root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 801d: dev pppoe-wan root refcnt 2 bandwidth 950Kbit diffserv3 dual-srchost nat nowash no-ack-filter split-gso rtt 100ms atm overhead 44
 Sent 26704551 bytes 303899 pkt (dropped 26517, overlimits 592826 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 362Kb of 4Mb
 capacity estimate: 950Kbit
 min/max network layer size:           32 /    1328
 min/max overhead-adjusted size:      106 /    1537
 average network hdr offset:            0

                   Bulk  Best Effort        Voice
  thresh       59368bit      950Kbit    237496bit
  target          307ms       19.2ms       76.7ms
  interval        614ms        114ms        172ms
  pk_delay          0us       39.9ms        186ms
  av_delay          0us       17.6ms        166ms
  sp_delay          0us         30us       5.36ms
  backlog            0b           0b           0b
  pkts                0       310892        19524
  bytes               0     38345788      8871343
  way_inds            0          402            0
  way_miss            0         4897           61
  way_cols            0            0            0
  drops               0        24695         1822
  marks               0            0            0
  ack_drop            0            0            0
  sp_flows            0            1            2
  bk_flows            0            1            0
  un_flows            0            0            0
  max_len             0         1278         1328
  quantum           300          300          300

qdisc ingress ffff: dev pppoe-wan parent ffff:fff1 ----------------
 Sent 617888249 bytes 539986 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 801e: dev ifb4pppoe-wan root refcnt 2 bandwidth 17Mbit besteffort dual-dsthost nat nowash ingress no-ack-filter split-gso rtt 100ms atm overhead 44
 Sent 612287817 bytes 535276 pkt (dropped 4710, overlimits 994408 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 222Kb of 4Mb
 capacity estimate: 17Mbit
 min/max network layer size:           28 /    1328
 min/max overhead-adjusted size:      106 /    1537
 average network hdr offset:            0

                  Tin 0
  thresh         17Mbit
  target            5ms
  interval        100ms
  pk_delay       2.56ms
  av_delay       1.13ms
  sp_delay         23us
  backlog            0b
  pkts           539986
  bytes       617888249
  way_inds        51782
  way_miss         3783
  way_cols            0
  drops            4710
  marks               0
  ack_drop            0
  sp_flows            1
  bk_flows            1
  un_flows            0
  max_len          1328
  quantum           518






ifstatus wan | grep -e \"device\"

 "device": "eth0.2",

about the packet capture I have made 4 examples with csgo and cs2
2 captures in csgo with no other network load and then with the stressor. and the same thing with cs2.
I have uploaded them to Mediafire and here is the link for you to analyze
note that I was not able to play on Spain servers that are the closest to my location because they are down and on maintenance, so I was connected to other servers in Germany.
https://www.mediafire.com/file/2628hpzuest3b6z/Captures.rar/file

moeller0 · September 16, 2023, 10:24am

I think you might want to try ACK-filtering, as you are really short on upload capacity, and ACK filtering can help a bit... so try

        option eqdisc_opts 'nat dual-srchost ack-filter'

Also there is uplink traffic in the Voice tin, but too much for that tin to carry (166ms av_delay is crazy high). So consider changing either:
a) from diffserv3 to besteffort, essentially using a single priority tier (do this if you do not use DSCPs for your game)
b) fro diffserv3 to diffserv4 and make sure game traffic is sorted into the Video not the Voice tin (and nothing in the Voice tin) Voice only gets 25% of the capacity as guaranteed priority traffic, while diffserv4's Video gets 50%, which might just be enough for your CS2 game....

But let me be honest here, it seems that modern games simply assume higher upload rates and hence you are in the territory where we can manage the pain better or worse, but things will always be a bit fickle simply because you are living really close to the edge here.

I am currently out of time for the next week, so will only be able to look into captures next weekend (or later). Thanks for the data.

lleachii · September 16, 2023, 7:46pm

Your video says:

To be clear:

You're purposely causing traffic to occur while you play?
If so, why?
What happens when you don't try to mess up the game by causing bandwidth shortages during game play?

Xsanchi · September 16, 2023, 8:11pm

@moeller0 I have tried the ack-filtering nothing has changed.
about changing the diffserv3 to besteffort or diffserv4, sadly I have no idea how to do that via router page or winscp or putty? is it an easy thing or a complicated one? would appreciate it if you could help me with that.
@lleachii Yeah, I purposely did that because that's what the sqm is for. to play online games without lag even if the bandwidth is being used?
about your last question, the game is just fine and playable if the internet is not being used by anything but only the game.

moeller0 · September 16, 2023, 8:19pm

Ah, sorry, same as ack-filter, just put either diffserv4 or or besteffort into
so change

        option eqdisc_opts 'nat dual-srchost'

to

        option eqdisc_opts 'nat dual-srchost diffserv4'

or

        option eqdisc_opts 'nat dual-srchost besteffort'

and tehn restart sqm (I hope you did this for your ack-filter test as well):

/etc/init.d/sqm stop ; /etc/init.d/sqm start

then always use tc -s qdisc to confirm that cake actually reports the desired configuration.

moeller0 · September 23, 2023, 8:37pm

So here is a quick look at the two CS versions with the stressor:
First CSGO:
CSGO Bytes/second:

CSGO Packets/second:

Second CS2:
CS2 Bytes/second:

CS2 Packets/second:

Interestingly, there were two massive drops (around the 45 and 75 seconds mark) that must habe effected game play, that where visible for all traffic.

But CS2 clearly upped the packet sizes somehow, both up- and download seem to run at at least twice the bytes/second rate than in CSGO, with your really slow uplink that might already be enough to make it quite fickle... in addition the packet rates for CSGO are pretty stable at ~60Hz for both directions (upload in green is super smooth, whie download shows some more variability, but IIRC that is mostly that occasionally the server sent 2 instead of one packet).
Now for CS2 the packet rates are not as smooth and seem to sit at odd values, download at ~100 Hz, upload at maybe 40Hz. This might, or might not be "sub-tick" or it might already be your problem, say if the game engines expects 60 Hz and it gets ~40Hz timing will likely be interesting.

Now I am only a casual wireshark user, so I might overlook things here, maybe @dlakelan might chime in, as he has way more experience with game traces....

CS2:

with stressor:
upload rate:    176 Kbps
download rate: 965 Kbps
without stressor:

upload rate:    375 Kbps
download rate: 1073 Kbps

CSGO:

with stressor
upload rate:     92.881 Kbps
download rate:  423.385 Kbps

without stressor
upload rate:     97 Kbps
download rate:  430 Kbps

EDIT:

looking at CS2 without the stressor, we see that it would like to run at ~64Hz in upload direction, so unless the observed ~40 Hz rate with the stressor is intended behaviour under congestion, I think this looks like the culprit potentially caused by your really small upload (when the stressor runs in the same host as your game, the per internal-IP isolation does not work anymore in your favor).
I guess if CS2 is what you want to play you should change your testing regime to have the stressor run from another computer to see whether cake's per internal IP fairness is good enough, but if CS2 really want like 1/3 of your upload you will need to make sure that not more than 2 other machines are actively using upload traffic, or consider using diffserv4 and only move your gaming UDP traffic to the VIDEO tin (and keep everything else in the BestEffort tin), this might work, as the VIDEO tin has priority access up to 50% of the configured capacity, which in your case would be ~500 Kbps which might be enough for making CS2 playable.