Help configuring new openwrt for gaming

Hello I am a newcomer to openWRT and would like some help. Apologies for the word wall in advance.

My setup consists of a modem (Ziggo connectbox set to bridge mode) with speeds of 1Gps down and 100mbps up that feeds to my openwrt router (raspberry pi 4 model b 4gb ram running OpenWrt 23.05.3 r23809-234f1a2efa / LuCI openwrt-23.05 branch git-24.086.45142-09d5a38). The router feeds to a tp-link deco m4 in ap-mode which connects to two other tp-link m4 boxes in different locations in my house. One of these locations is a garden room that's disconnected from the house where my gaming computer and server are. Prior to setting up the raspberrypi as a router I was using the deco boxes and not having issues with ping on my gaming computer so I feel confident that its a router settings issue and not a connection outside my house issue but would be happy to be wrong :stuck_out_tongue: .

I've been reading everything I can on the forum and have come away confused on what the optimal setup is for online gaming and openwrt. Some of the things I've seen include:

  • recommendations to setup sqm. Which I have done according to the raspberry pi setup guide I followed adjusting to accommodate my internet speeds. That helped bufferbloat go from a B to an A but didn't improve lag in my games.
  • I found this settings post by a user mindwolf where they set up an rc.local file, sysctl.conf, firewall.user, and tc settings. I was confused on what the tc command was doing. I was able to implement the file settings (rc, sysctl, and firewall) but those had no impact on lag while gaming.
  • I also saw this post by Dopam-IT_1987 - My configuration gaming for the moment but am unsure how to setup the chain postrouting settings or if it would even help at this point.
  • I've also seen mentions of a gaming script setup by dlakelan that may or may not work anymore.
  • Along with various firewall edits and things users have done that I can't seem to find on hand at the moment but will update when I can.
  • oh and qosify? That may or may not work anymore too I'm not sure.

So all that to say as a baby openWRT user who has no clue what they're doing I'm lost. I know there isn't one way to setup a router and everyone has different circumstances but I'm not sure where to even begin. Is there a resource or guide I'm missing?

TLDR: new openWRT user lost on where to begin optimizing router for gaming

Run some tests:

  • cable connected as close to internet as possible
  • wireless as far as you might need it
  • some steps in the middle.

Hey I have compiled 45 or so buffer bloat tests 30 were on my laptop connected directly to the raspberry pi, and the other 15 were from my gaming desktop. Below are the average values for each test run

SQM Connection Unloaded (ms) Download (+ms) download speed (Mbps) Upload (+ms) upload speed (Mbps)
on laptop direct to the raspberrypi 13.56 0.59 840.49 0.00 83.44
off laptop direct to the raspberrypi 13.61 0.56 839.91 0.00 83.48
on gaming pc 2 switches away 13.65 0.52 840.89 0.00 83.51

Interestingly this morning I tried playing the same game and wasn't having ping issues. Maybe it was a fluke but there are a few significant things that have changed that I can think of since yesterday, my partner is at work and not using their computer, the time of day (morning vs night), and I removed one switch from the network. One or all of these could be the culprit so I will test to see if it happens again tonight when my partner is online / watching netflix and see if my ping issues happen again.

In another interesting turn of events it seems having openwrt has slightly improved my average download speeds but lowered my upload speeds. I run a speedtest hourly in a docker container

desc # tests avg ping (ms) avg download (Mbps) avg upload (Mbps)
prior to openwrt 319 8.1174670846395 637.998370382445 94.3749544075235
openwrt only 278 8.56087410071942 781.629375654676 88.8406021007194
combined 597 8.32394472361809 704.881820073702 91.7978188274707

Not that this has any impact on this post but some fun data :smiley:

That partially implies that your home network might not be where the issue arises and hence little you can do to fix it there...

I would recommend to take a step back and revert to the default settings and if performance is not good enough I would step by step re introduce these changes and see which actuall improve things...
One thing I would recommend, given that you share the network with non-gaming machines/users, is to try cake's per internal IP isolation mode, so your gaming traffic does not get hurt by your partner's internet usage (however game traffic is relatively light, so even simple par flow fairness should do the right thing out of the box).

Your tests however seem to imply that SQM does not improve your performance under load (but I would need to see the output of tc -s qdisc to assess whether cake actually engaged here)...

With or without SQM, and with which SQM settings?

ifstatus wan | grep -e device
cat /etc/config/sqm

The overall average is a mix though the majority of those tests will be with SQM on since I've had that turned on for the whole time I've been using openWRT other than maybe a half day where I was testing it. So as a rough guess maybe 12-15 of those tests in the openwrt only number will be without SQM in them.

Yeah I should probably just leave things as they are from a fresh install and see what the network performance is like. I should have thought ahead and made a configuration backup to right after I set the router up so I wouldn't have to reflash the whole thing.

those commands return the following:

harmodius@fedora:~$ ssh root@

BusyBox v1.36.1 (2024-06-08 15:30:58 UTC) built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 OpenWrt 23.05.3, r23809-234f1a2efa
root@OpenWrt:~# ifstatus wan | grep -e device
	"l3_device": "eth1",
	"device": "eth1",
root@OpenWrt:~# cat /etc/config/sqm

config queue 'eth1'
	option enabled '1'
	option interface 'eth1'
	option download '900000'
	option upload '90000'
	option qdisc 'cake'
	option script 'piece_of_cake.qos'
	option linklayer 'ethernet'
	option debug_logging '0'
	option verbosity '5'
	option overhead '42'

That returns quite a lot of text but I think I can figure out how to do a textbox

tc -s qdisc output
root@OpenWrt:~# tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc mq 0: dev eth0 root 
 Sent 210118696691 bytes 158996381 pkt (dropped 23841, overlimits 0 requeues 13829552) 
 backlog 0b 0p requeues 13829552
qdisc fq_codel 0: dev eth0 parent :5 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64 
 Sent 43519494974 bytes 32964771 pkt (dropped 1905, overlimits 0 requeues 5220747) 
 backlog 0b 0p requeues 5220747
  maxpacket 27252 drop_overlimit 0 new_flow_count 560448 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :4 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64 
 Sent 46546658698 bytes 35224864 pkt (dropped 433, overlimits 0 requeues 2124816) 
 backlog 0b 0p requeues 2124816
  maxpacket 25738 drop_overlimit 0 new_flow_count 805519 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :3 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64 
 Sent 40260492377 bytes 30247862 pkt (dropped 127, overlimits 0 requeues 2414373) 
 backlog 0b 0p requeues 2414373
  maxpacket 27252 drop_overlimit 0 new_flow_count 1000768 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64 
 Sent 44011141469 bytes 33722339 pkt (dropped 1934, overlimits 0 requeues 3729616) 
 backlog 0b 0p requeues 3729616
  maxpacket 27252 drop_overlimit 1536 new_flow_count 961592 ecn_mark 0 drop_overmemory 1536
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64 
 Sent 35780909173 bytes 26836545 pkt (dropped 19442, overlimits 0 requeues 340000) 
 backlog 0b 0p requeues 340000
  maxpacket 27252 drop_overlimit 8960 new_flow_count 473714 ecn_mark 0 drop_overmemory 8960
  new_flows_len 0 old_flows_len 0
qdisc cake 8011: dev eth1 root refcnt 2 bandwidth 90Mbit besteffort triple-isolate nonat nowash no-ack-filter split-gso rtt 100ms noatm overhead 42 
 Sent 6966737748 bytes 13687174 pkt (dropped 15241, overlimits 17367100 requeues 1296) 
 backlog 0b 0p requeues 1296
 memory used: 1139484b of 4500000b
 capacity estimate: 90Mbit
 min/max network layer size:           28 /    1500
 min/max overhead-adjusted size:       70 /    1542
 average network hdr offset:           14

                  Tin 0
  thresh         90Mbit
  target            5ms
  interval        100ms
  pk_delay          6us
  av_delay          3us
  sp_delay          2us
  backlog            0b
  pkts         13702415
  bytes      6988829533
  way_inds       116038
  way_miss        66327
  way_cols            0
  drops           15241
  marks               0
  ack_drop            0
  sp_flows            2
  bk_flows            1
  un_flows            0
  max_len         12112
  quantum          1514

qdisc ingress ffff: dev eth1 parent ffff:fff1 ---------------- 
 Sent 51896214191 bytes 40267009 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-lan root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc cake 8012: dev ifb4eth1 root refcnt 2 bandwidth 900Mbit besteffort triple-isolate nonat wash no-ack-filter split-gso rtt 100ms noatm overhead 42 
 Sent 52860607694 bytes 39796726 pkt (dropped 470283, overlimits 34383764 requeues 0) 
 backlog 0b 0p requeues 0
 memory used: 9465612b of 15140Kb
 capacity estimate: 900Mbit
 min/max network layer size:           46 /    1500
 min/max overhead-adjusted size:       88 /    1542
 average network hdr offset:           14

                  Tin 0
  thresh        900Mbit
  target            5ms
  interval        100ms
  pk_delay         15us
  av_delay          5us
  sp_delay          1us
  backlog            0b
  pkts         40267009
  bytes     53565070477
  way_inds       472192
  way_miss        65557
  way_cols            0
  drops          470283
  marks               0
  ack_drop            0
  sp_flows            3
  bk_flows            1
  un_flows            0
  max_len         68130
  quantum          1514

edit: clarify that the sqm tests only apply to the openwrt average number

Would this look like setting up a virtual lan connection with just my desktop pc and then another with everything else?

with your settings you can at best expect the following goodput (online tests typically return typically TCP payload rate):
900 * ((1500-20-20) / (1500+42)) = 852.14 Mbps
90 * ((1500-20-20) / (1500+42)) = 85.21 Mbps

So either your upload test returns non-payload rates or this is just the typical imprecision of online tests...

No, it would look like now, but cake would as a first round share available capacity equitably between active internal IP addresses (if say PC1 only actually uses 10% of capacity, PC2 would still get the remaining 90% so there is nothing wasted, but with two active PCs the 'guaranteed' capacity would be 50% for each PC).

To actually configure this, try the following /etc/config/sqm:

config queue 'eth1'
	option ingress_ecn 'ECN'
	option egress_ecn 'ECN'
	option itarget 'auto'
	option etarget 'auto'
	option verbosity '5'
	option qdisc 'cake'
	option qdisc_advanced '1'
	option squash_dscp '0'
	option squash_ingress '0'
	option qdisc_really_really_advanced '1'
	option linklayer 'ethernet'
	option linklayer_advanced '1'
	option tcMTU '2047'
	option tcTSIZE '128'
	option linklayer_adaptation_mechanism 'default'
	option debug_logging '1'
	option interface 'eth1'
	option tcMPU '84'
	option enabled '1'
	option overhead '42'
	option iqdisc_opts 'dual-dsthost ingress memlimit 32mb'
	option eqdisc_opts 'dual-srchost memlimit 32mb ack-filter'
	option script 'piece_of_cake.qos'
	option upload '90000'
	option download '900000'

This will do the following:
option tcMPU '84': make sure small ACK-only packets are accounted correctly
option iqdisc_opts 'dual-dsthost ingress memlimit 32mb': make sure ingress/download capacity is shared by the destination IP address first, the shaper will try to shape the incoming rate, not the outgoing rate (which helps in adjusting to under responsive flows a bit), it will also increase the worst case memory for queueing, as openwrt's default is not ideal for long RTT flows on fast links like yours option eqdisc_opts 'dual-srchost memlimit 32mb ack-filter'`: make sure egress/upload capacity is shared by the source IP address first, increase the worst case memory for queueing, as openwrt's default is not ideal for long RTT flows on fast links like yours, also remove some redundant ACK packets smoothing out the somewhat bursty ACK flow on a docsis link and recovering some upload capacity otherwise used up for ACK traffic for download TCP data.

You could try the isolation mode by: running two concurrent speedtests on your gaming computer and one on your partner's computer, without the dual-xxxhost modes each oof the three tests should give you roughly 1/3 of your capacity, with the dual-xxxhost keywords in place and active the two speedtests on the same computer will only get ~1/4 of the total capacity so that the sum of tests on both machines comes out about equal...

For many home networks this policy, while not optimal, is 'good enough'....

I'll have to mull this over haha its a lot of information on network knowledge I'm not up to speed on. I appreciate the reply and will come back after I figure it out with an update!

I wanted to know how this IP isolation is done

See here:

Especially the section titled "B. Flow Isolation and Hashing"

Just one question, go and see this article which is in PDF

Not sure what your exact point is, the link also carries the latex source for the article, so not just pdf. Anyway here is the relevant section:

B. Flow Isolation and Hashing

CAKE replaces the direct hash function used in FQ-CoDel with an 8-way set-associative hash. While set-associative hash- ing has been well-known for decades as a means to improve the performance of CPU caches [22], it has not seen much use in packet scheduling. Conceptually, a k−way set-associative hash with n total buckets can be thought of as a plain hash with n/k buckets that is only considered to have a collision if more than k items hash into the same bucket. As can be seen in Figure 1, this significantly reduces the hash collision probability up to the point where the number of flows is larger than the number of queues.2

  1. Host Isolation: With flow fairness, hosts can increase their share of the available bandwidth by splitting their traffic over multiple flows. This can be prevented by providing host fairness at the endpoint IP address level, which CAKE can do in addition to flow fairness.

The host isolation is simple in concept: The effective DRR quantum is divided by the number of flows active for the flow endpoint. This mechanism can be activated in three different modes: source address fairness, in which hosts on the local LAN receive equal share, destination address fairness, in which servers on the public internet receive an equal share, or "triple isolate" mode, in which the maximum of the source and destination scaling is applied to each flow. CAKE also hooks into the Linux kernel Network Address Translation (NAT) subsystem to obtain the internal host address of a packet, which would otherwise be obscured since packets are queued after NAT is applied.

CAKE accomplishes this scaling as shown in Algorithm 2: When a packet is enqueued it is hashed into a queue using the transport layer port numbers along with the source and destination IP addresses. In addition, two separate hashes are performed on the packet destination IP address and source IP address. A separate set of hash buckets is kept for these address hashes. These buckets do not contain a queue of packets, but instead a data structure that keeps two reference counts for each IP address, which track the number of active flows with the given address as source and destination, respectively.

The per-IP reference counts are used to modify the quantum for each active flow. When a flow is scheduled, its "host load" is calculated as the maximum of the reference counts for its source and destination IP addresses. The effective quantum of the flow is simply divided by this load value, which achieves the desired scaling.

Straight from cake's authors :wink:

Please disregard that old post of mine entirely. I've learned a lot since then and it would also vary depending on your situation. FYI: DSCP marking does not make a difference 99% outside of your LAN, unless you control the endpoints and everything inbetween.

1 Like

You can set meta priority, dscp value without altering packets on the wire. Then you need qdisc which takes priorities like pfifo_fast.

I agree, but setting priorities internally won't affect the priority of packets traveling across the WAN as many might expect. ISPs typically reclassify packets as CS1 or CS0 unless there is some type of priority elevation, such as VoIP traffic on a specific VLAN under a business contract. Additionally, your packets will still be subject to the ISP's queues and latency, which is generally not sub-1ms in most cases. A single packet inside data centers typically traverses numerous overloaded cloud fabrics, VMs, firewalls, switches, application filters, and other network elements. This is the common scenario in the USA today.