Software Flow offloading not working on AIoT AC2350 Xiaomi

Hi guys, here people stated that with Software Flow offloading they managed to achieve 700+Mbps on a gigabit port. I'm trying to do the same, however, can't get it work.
My setup is this:
root@Xiaomi-TV:~# cat /proc/cpuinfo

system type		: Qualcomm Atheros QCA956X ver 1 rev 0
machine			: Xiaomi AIoT AC2350
processor		: 0
cpu model		: MIPS 74Kc V5.0
BogoMIPS		: 385.84
wait instruction	: yes
microsecond timers	: yes
tlb_entries		: 32
extra interrupt vector	: yes
hardware watchpoint	: yes, count: 4, address/irw mask: [0x0ffc, 0x0ffc, 0x0ffb, 0x0ffb]
isa			: mips1 mips2 mips32r1 mips32r2
ASEs implemented	: mips16 dsp dsp2
Options implemented	: tlb 4kex 4k_cache prefetch mcheck ejtag llsc dc_aliases perf_cntr_intr_bit cdmm contextconfig perf
shadow register sets	: 1
kscratch registers	: 0
package			: 0
core			: 0
VCED exceptions		: not available
VCEI exceptions		: not available

root@Xiaomi-TV:~# cat /etc/openwrt_release

DISTRIB_ID='OpenWrt'
DISTRIB_RELEASE='22.03.0-rc4'
DISTRIB_REVISION='r19426-2b1941e47d'
DISTRIB_TARGET='ath79/generic'
DISTRIB_ARCH='mips_24kc'
DISTRIB_DESCRIPTION='OpenWrt 22.03.0-rc4 r19426-2b1941e47d'
DISTRIB_TAINTS=''
NAME="OpenWrt"
VERSION="22.03.0-rc4"
ID="openwrt"
ID_LIKE="lede openwrt"
PRETTY_NAME="OpenWrt 22.03.0-rc4"
VERSION_ID="22.03.0-rc4"
HOME_URL="https://openwrt.org/"
BUG_URL="https://bugs.openwrt.org/"
SUPPORT_URL="https://forum.openwrt.org/"
BUILD_ID="r19426-2b1941e47d"
OPENWRT_BOARD="ath79/generic"
OPENWRT_ARCH="mips_24kc"
OPENWRT_TAINTS=""
OPENWRT_DEVICE_MANUFACTURER="OpenWrt"
OPENWRT_DEVICE_MANUFACTURER_URL="https://openwrt.org/"
OPENWRT_DEVICE_PRODUCT="Generic"
OPENWRT_DEVICE_REVISION="v0"
OPENWRT_RELEASE="OpenWrt 22.03.0-rc4 r19426-2b1941e47d"

/etc/config/firewall:

config defaults
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option synflood_protect '1'
	option flow_offloading '1'
#	option flow_offloading_hw '1'

By default nft_flow_offload was not installed, so I installed it:
root@Xiaomi-TV:~# opkg list-installed|grep offload

kmod-ipt-offload - 5.10.120-1
kmod-nft-offload - 5.10.120-1

root@Xiaomi-TV:~# lsmod |grep nf

libcrc32c                592  1 nf_tables
nf_conntrack           69296  7 nft_redir,nft_nat,nft_masq,nft_flow_offload,nft_ct,nf_nat,nf_flow_table
nf_defrag_ipv4          1216  1 nf_conntrack
nf_defrag_ipv6          5584  1 nf_conntrack
nf_flow_table          22416  5 xt_FLOWOFFLOAD,nf_flow_table_ipv6,nf_flow_table_ipv4,nf_flow_table_inet,nft_flow_offload
nf_flow_table_inet       576  1 
nf_flow_table_ipv4       448  0 
nf_flow_table_ipv6       448  0 
nf_log_common           2912  2 nf_log_ipv6,nf_log_ipv4
nf_log_ipv4             3584  0 
nf_log_ipv6             3904  0 
nf_nat                 22032  4 nft_redir,nft_nat,nft_masq,nft_chain_nat
nf_reject_ipv4          2560  3 ipt_REJECT,nft_reject_ipv4,nft_reject_inet
nf_reject_ipv6          2880  2 nft_reject_ipv6,nft_reject_inet
nf_tables             147680154 nft_fib_inet,nf_flow_table_ipv6,nf_flow_table_ipv4,nf_flow_table_inet,nft_reject_ipv6,nft_reject_ipv4,nft_reject_inet,nft_reject,nft_redir,nft_quota,nft_objref,nft_numgen,nft_nat,nft_masq,nft_log,nft_limit,nft_hash,nft_flow_offload,nft_fib_ipv6,nft_fib_ipv4,nft_fib,nft_ct,nft_counter,nft_chain_nat
nfnetlink               5152  1 nf_tables
nft_chain_nat            864  2 
nft_counter             1728 13 
nft_ct                  7648  3 
nft_fib                 1408  3 nft_fib_inet,nft_fib_ipv6,nft_fib_ipv4
nft_fib_inet             672  0 
nft_fib_ipv4            2080  1 nft_fib_inet
nft_fib_ipv6            2784  1 nft_fib_inet
nft_flow_offload        3456  1 
nft_hash                2400  0 
nft_limit               3456  5 
nft_log                 1664  0 
nft_masq                1824  1 
nft_nat                 2624  0 
nft_numgen              1680  0 
nft_objref              1664  0 
nft_quota               1952  0 
nft_redir               1792  0 
nft_reject              1120  3 nft_reject_ipv6,nft_reject_ipv4,nft_reject_inet
nft_reject_inet         1152  2 
nft_reject_ipv4          672  0 
nft_reject_ipv6          672  0

It seems all is setup correctly, however, after restarting firewall
root@Xiaomi-TV:~# iperf3 -c 172.16.66.9

Connecting to host 172.16.66.9, port 5201
[  5] local 172.16.66.215 port 41714 connected to 172.16.66.9 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.08   sec  15.9 MBytes   124 Mbits/sec    0   66.5 KBytes       
[  5]   1.08-2.02   sec  15.0 MBytes   133 Mbits/sec    0   66.5 KBytes       
[  5]   2.02-3.01   sec  20.0 MBytes   169 Mbits/sec    0   66.5 KBytes       
[  5]   3.01-4.01   sec  18.8 MBytes   158 Mbits/sec    0   66.5 KBytes       
[  5]   4.01-5.00   sec  20.0 MBytes   169 Mbits/sec    0   66.5 KBytes       
[  5]   5.00-6.02   sec  18.8 MBytes   154 Mbits/sec    0   66.5 KBytes       
[  5]   6.02-7.06   sec  17.5 MBytes   141 Mbits/sec    0   66.5 KBytes       
[  5]   7.06-8.04   sec  15.0 MBytes   128 Mbits/sec    0   66.5 KBytes       
[  5]   8.04-9.04   sec  20.0 MBytes   168 Mbits/sec    0   66.5 KBytes       
[  5]   9.04-10.04  sec  20.0 MBytes   168 Mbits/sec    0   66.5 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.04  sec   181 MBytes   151 Mbits/sec    0             sender
[  5]   0.00-10.04  sec   181 MBytes   151 Mbits/sec                  receiver

And cpu is 100% while testing.
What did I miss?

Software Flow offloading may improve WAN/LAN performance, but it will still be limited by the CPU capacity.

Your device has a SoC with a single-core CPU MIPS running at 775Mhz (slow). You are not likely to achieve any throughput higher than what you are observing, even with Software Flow offloading.

You will only get Gigabit WAN performance if using dual or quad core ARM or x86 with a higher clock.

Alternatively you can find a Mediatec mt7621/7622 device, which support hardware flow offload. On these devices you can achieve Gigabit WAN performance (using HW flow offload) but only if you do not use SQM.

In this case how do those guys got 925.71/591.88 Mbps with speedtest a year ago? Something goes wrong now with the offloading process. It just doesn't work and I'd like to fix it.

I managed to sort it out. Partly. The thing is, that offloading works ONLY for packets forwarding, meaning wan <-> lan traffic. In this case I can get about 800Mbps both directions. However, if I connect my mobile to this AP and try to reach iperf3 server in the lan I have about 150Mbps speed. This is because there is no offloading for the packets from wlan to lan, as they are bridged together in br-lan. How can I modify rules to make offloading work in this case too?

1 Like

How did you do this mate? I can't exceed 200mbit at the moment and on a wired connection. download is not working.