# How does (software) flow\_offloading interact with SQM? **URL:** \ **Category:** Installing and Using OpenWrt\ **Created:** [December 11, 2021, 1:57am UTC](https://forum.openwrt.org/t/how-does-software-flow-offloading-interact-with-sqm/114310 "2021-12-11T01:57:59Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![RollingStar](https://forum.openwrt.org/user_avatar/forum.openwrt.org/rollingstar/32/38227_2.png) [@RollingStar](https://forum.openwrt.org/u/RollingStar)\ **Post date:** [December 11, 2021, 1:57am UTC](https://forum.openwrt.org/t/how-does-software-flow-offloading-interact-with-sqm/114310/1 "2021-12-11T01:57:59Z")
Lots of conflicting info online. > [@Software flow offloading implications](https://forum.openwrt.org/t/software-flow-offloading-implications/90957/26): > > with sqm no sfo [http://www.dslreports.com/speedtest/68678992](http://www.dslreports.com/speedtest/68678992) root@OpenWrt:~# tc -s -d qdisc qdisc noqueue 0: dev lo root refcnt 2 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) backlog 0b 0p requeues 0 qdisc fq\_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum 1518 target 5ms interval 100ms memory\_limit 4Mb ecn drop\_batch 64 Sent 144618747286 bytes 174506922 pkt (dropped 0, overlimits 0 requeues 27) backlog 0b 0p requeues 27 maxpacket 9108 drop\_overlimit 0 new\_flow\_… > [@Thanks for software flow offloading](https://forum.openwrt.org/t/thanks-for-software-flow-offloading/49760/3): > > There is a warning at least on 18.06.5: "Experimental feature. Not fully compatible with QoS/SQM. " Anyone using / knowing more about? > Well software flow offloading (or any accellerator technology for that matter) bypasses various kernel subsystems to achieve higher overall throughput. This conflicts with shapers and queue management to some extent as these need access to all packets of a flow and not just the initial handshake ones. Is this correct? Can someone give a high level summary of how flow offloading works as opposed to SQM? Can they coexist at all in openWRT? I did some informal testing and found that SQM (fq\_codel) "alongside" software flow offloading (`flow_offloading` not `flow_offloading_hw`) raised my downstream from around 120 Mbps to 140 Mbps. However it seemed to come at a cost of 0-15 ms of additional bufferbloat latency vs fq\_codel on its own. In addition, I'm still not sure if fq\_codel was active whatsoever, based on my reading that they "conflict" with each other. My bufferbloat tests are far better with "both" active than with both disabled - my loaded latency was 200-400ms before enabling them. No offloading: [https://www.waveform.com/tools/bufferbloat?test-id=9ca25e80-c8a5-4913-8709-b3a03a1525b3](https://www.waveform.com/tools/bufferbloat?test-id=9ca25e80-c8a5-4913-8709-b3a03a1525b3) Yes offloading: [https://www.waveform.com/tools/bufferbloat?test-id=53792090-4746-4579-8c7a-5501fedf3956](https://www.waveform.com/tools/bufferbloat?test-id=53792090-4746-4579-8c7a-5501fedf3956) More reading: [https://www.reddit.com/r/openwrt/comments/fzvt9s/sqm\_qos\_flow\_offloading/](https://www.reddit.com/r/openwrt/comments/fzvt9s/sqm_qos_flow_offloading/) ---
**Author:** ![StepSis](https://forum.openwrt.org/user_avatar/forum.openwrt.org/stepsis/32/67753_2.png) [@StepSis](https://forum.openwrt.org/u/StepSis)\ **Post date:** [March 31, 2023, 11:10am UTC](https://forum.openwrt.org/t/how-does-software-flow-offloading-interact-with-sqm/114310/2 "2023-03-31T11:10:32Z")
Wish this would see more development ... 😶 I found couple of interesting threads: > [@Software flow offloading implications](https://forum.openwrt.org/t/software-flow-offloading-implications/90957/29): > > I have to admit that I have no reliable information to share. However as far as I understand, software offloading should work with SQM as it tries to avoid other parts of the network stack higher up than mere qdiscs like sqm uses. Hardware flow offloading however will hide all packets from sqm, so the hardware offloading engine needs to offer its own qdiscs, like the NSS stuff on e.g. r7800 as far as I understand. > [@Software flow offloading implications](https://forum.openwrt.org/t/software-flow-offloading-implications/90957/30): > > It's working if the SQM interface set as wan (device), but software flow offloading does not help with SQM if the interface set as pppoe-wan (tunnel). My test on my friend's FTTH line with 200/20 plan showed that Xiaomi Mi Router 4A Gigabit Edition is able to shape up to 150mbps using CAKE without software flow offloading and it's fine doing 200mbps with software flow offloading turned on. But I'm afraid Diffserv is not working when shaping is done on wan instead of pppoe-wan, every packet goes … > [@SQM and HTB hardware offload](https://forum.openwrt.org/t/sqm-and-htb-hardware-offload/112046): > > Since kernel 5.12 HTB qdisc has an API for hardware offload. Mellanox is using this feature. Would it make sense to try and implement this for some of our targets? Using Hardware HTB in combination with software fq\_codel could allow us to do SQM at higher rates? I am not a developer but if there is any reading this , what do you think about this? We are approaching territory where SQM becomes impossible on all but highest end hardware... You need a beast of a router to work with gigabit ethernet (Let's not even talk about higher speeds - 5 or even 10 gigabit, straight up impossible) ---
**Author:** ![moeller0](https://forum.openwrt.org/user_avatar/forum.openwrt.org/moeller0/32/1591_2.png) [@moeller0](https://forum.openwrt.org/u/moeller0)\ **Post date:** [March 31, 2023, 12:01pm UTC](https://forum.openwrt.org/t/how-does-software-flow-offloading-interact-with-sqm/114310/3 "2023-03-31T12:01:21Z")
> [@StepSis](#): > > You need a beast of a router to work with gigabit ethernet (Let's not even talk about higher speeds - 5 or even 10 gigabit, straight up impossible) Not really: a) most ISP routers are shipped with wholly under powered CPUs for the links they are deployed on, and make up for that by using some off-load engines. Already relatively old x86 devices manages a gigabit in software quite well and are not that power hungry. But sure most convenient all-in-one routers are out of their league for such fast link rates. b) 5-10 Gbps, honestly, why does one need that right now? Besides folks constantly sinking and sourcing large bodies of data (this is the one application ), I see little practical improvement from switch from 100/40 to 250/40, let alone to 1000 or even "10000" Mbps. Well, it might give you bragging rights, but for that it really does not matter whether that contracted speed is actually usable 😉 Personally I used my 100/40 link for a long time shaped down to 49/29 IIRC as that was the SQM limit for my old router; in my testing for my work-loads, sacrificing ~50% of the aggregate link speed was less of an issue than not having SQM. I ended up switching to another router that has no problems with SQM at 100/40, but my point is just because a contract says X Mbps there is no requirement to actually run the shaper closer t that rate. One might even able to save some money, by ordering a rate that is within one's router's SQM "limit". But that clearly is subjective, if for your use-cases raw rate is more important than acceptable responsiveness under load I am happy to accept that. ---
**Author:** ![StepSis](https://forum.openwrt.org/user_avatar/forum.openwrt.org/stepsis/32/67753_2.png) [@StepSis](https://forum.openwrt.org/u/StepSis)\ **Post date:** [March 31, 2023, 1:18pm UTC](https://forum.openwrt.org/t/how-does-software-flow-offloading-interact-with-sqm/114310/4 "2023-03-31T13:18:14Z")
> [@moeller0](#): > > a) most ISP routers are shipped with wholly under powered CPUs for the links they are deployed on, and make up for that by using some off-load engines. How are ISPs able to do SQM on their weak routers... is the hardware special in any way ? Like dedicated chips for hardware SQM or are they using same hardware that we all have in our routers? I think this should be the focus of development going forward... We are already struggling with gigabit speeds - pinned post being buy new router if you wanna do SQM ... Once everyone is on 5 to 10 gigabit... only x86\_64 will be able to do that? > [@moeller0](#): > > One might even able to save some money, by ordering a rate that is within one's router's SQM "limit" You already can't in my country 😛 Multiple ISPs offering 2Gbps as the cheapest package available. ---
**Author:** ![moeller0](https://forum.openwrt.org/user_avatar/forum.openwrt.org/moeller0/32/1591_2.png) [@moeller0](https://forum.openwrt.org/u/moeller0)\ **Post date:** [March 31, 2023, 2:17pm UTC](https://forum.openwrt.org/t/how-does-software-flow-offloading-interact-with-sqm/114310/5 "2023-03-31T14:17:05Z")
> [@StepSis](#): > > How are ISPs able to do SQM on their weak routers... is the hardware special in any way ? Well, most ISPs simply do not offer SQM at all... for others like the NSS cores there are at least fq-codel & shaper implementations that allow some sort of SQM configuration to use the offload engine (but the NSS offload system uses two general purpose CPUs and not dedicated ASICs for specific network functionality). > [@StepSis](#): > > Like dedicated chips for hardware SQM or are they using same hardware that we all have in our routers? Yeah that as far as I know does not really exist in that form (I hope I am simply misinformed). HOWEVER the most costly part of SQM is the actual traffic shaping, if that could be off-loaded generically to hardware and still combined with a software qdisc that would go a long way... > [@StepSis](#): > > I think this should be the focus of development going forward... Sure, go ahead and develop in that direction 😉 I am sure you will find a lot of interested users... > [@StepSis](#): > > We are already struggling with gigabit speeds - pinned post being buy new router if you wanna do SQM ... Once everyone is on 5 to 10 gigabit... only x86\_64 will be able to do that? Well, nobody forces you to either buy a 5-10 Gbps link and even if you do you can operate your traffic shaper at lower speeds.... I would guess that apples M1/M2 CPUs or amazons graviton2 would have little problem with 5-10 Gbps, but these likely are way more expensive than x86 solutions (unless you need multi-dozend core xeoan CPUs these are pretty pricy themselves). > [@StepSis](#): > > You already can't in my country 😛 Multiple ISPs offering 2Gbps as the cheapest package available. Well, OK, but you still can operate a traffic shaper at sat 100/100 even on a 2 Gbps link 😉