Adding OpenWrt support for Xiaomi AX3600 (Part 1)

dchard · 2021-11-17T11:48:17.907Z

Is this public? Let me take a look.

robimarko · 2021-11-17T11:49:51.231Z

I think you know the answer to that, you are gonna have to wait until they make a SPF release and then it will all get pushed out

Gruntruck · 2021-11-17T11:51:12.804Z

Gingernut:

Load seems to be better spread over cores but the actual transfer speed didn't improve.

Hi,

You cannot benchmark WiFi speed by copying files between NAS and Windows laptop. First, there is CIFS overhead. Second, NAS might not be able to push data faster. Please use iperf3 between two devices.

Gingernut · 2021-11-17T11:52:03.982Z

Without the smp_affinity_settings the transfer was loaded only on core0 and that more or less maxed out.

I was just doing some tests as some users asked.

robimarko · 2021-11-17T11:53:17.545Z

Sure, I was reffering to the case where no core was maxed out

Gingernut · 2021-11-17T11:56:14.869Z

You are right but as a quick test I usually do it this way.

BTW the exact same wireless transfer maxes out gigabit on stock firmware.

stock1621×601 77.9 KB

Stock interrupts just for the interested

root@XiaoQiang:~# cat /proc/interrupts
           CPU0       CPU1       CPU2       CPU3
  3:     566786      98541      90428    3341065       GIC  20 Edge      arch_timer
 40:          1          0          0          0   msmgpio  34 Edge      soc:gpio_keys
 76:          0          0          0          0       GIC 270 Level     bam_dma
 77:        396          0          0          0       GIC 340 Level     msm_serial0
 79:          2          0          0          0       GIC 127 Level     78b5000.spi
 80:          0          0          0          0       GIC 357 Edge      q6_wdog_interrupt
 81:          0          0          0          0       GIC 344 Edge      7803000.sdcc1ice
 82:          3          0          0          0       GIC 354 Edge      smp2p
 83:          0          0          0          0       GIC 276 Edge      tzerror
 88:     134610          0          0          0       GIC 409 Edge      nss_empty_buf_sos
 89:      49247          0          0          0       GIC 410 Edge      nss_empty_buf_queue
 90:          0          0          0          0       GIC 411 Edge      nss-tx-unblock
 91:    7046628          0          0          0       GIC 412 Edge      nss_queue0
 92:          0       4603          0          0       GIC 413 Edge      nss_queue1
 93:          0          0       4503          0       GIC 414 Edge      nss_queue2
 94:          0          0          0      61867       GIC 415 Edge      nss_queue3
 95:          0          0          0          0       GIC 416 Edge      nss_coredump_complete
 96:          0          0          0          0       GIC 417 Edge      nss_paged_empty_buf_sos
 97:      62821          0          0          0       GIC 422 Edge      nss_empty_buf_sos
 98:          0          0          0          0       GIC 423 Edge      nss_empty_buf_queue
 99:          0          0          0          0       GIC 424 Edge      nss-tx-unblock
100:    6544252          0          0          0       GIC 425 Edge      nss_queue0
101:          0          0          0          0       GIC 426 Edge      nss_queue1
102:          0          0          0          0       GIC 427 Edge      nss_queue2
103:          0          0          0          0       GIC 428 Edge      nss_queue3
104:          0          0          0          0       GIC 429 Edge      nss_coredump_complete
105:          0          0          0          0       GIC 430 Edge      nss_paged_empty_buf_sos
106:          0          0          0          0       GIC  35 Level     watchdog bark
107:         88          0          0          0       GIC 353 Edge      qcom,glink-smem-native-xprt-modem
108:          0          0          0          0       GIC 239 Level     bam_dma
109:     100429          0          0          0       GIC 178 Level     bam_dma
111:          0          0          0          0       GIC  84 Edge      qcom-pcie-msi
132:          4          0          0          0       GIC 348 Edge      ce0
133:     622897          0          0          0       GIC 347 Edge      ce1
134:     135587          0          0          0       GIC 346 Edge      ce2
135:       7790          0          0          0       GIC 343 Edge      ce3
137:         60          0          0          0       GIC 443 Edge      ce5
139:       8323          0          0          0       GIC  72 Edge      ce7
140:   20467709          0          0          0       GIC  71 Edge      ce8
141:          0          0          0          0       GIC 334 Edge      ce9
142:          0          0          0          0       GIC 333 Edge      ce10
143:          0          0          0          0       GIC  69 Edge      ce11
147:          8          0          0          0       GIC 326 Edge      host2rxdma-monitor-ring3
148:          0          0          0          0       GIC 325 Edge      host2rxdma-monitor-ring2
149:          8          0          0          0       GIC 324 Edge      host2rxdma-monitor-ring1
150:          0          0          0          0       GIC 323 Edge      reo2ost-exception
151:          0          0          0          0       GIC 322 Edge      wbm2host-rx-release
152:        160          0          0          0       GIC 321 Edge      reo2host-status
153:          0          0          0          0       GIC 320 Edge      reo2host-destination-ring4
154:          0          0          0          0       GIC 271 Edge      reo2host-destination-ring3
160:          1      87577          0          0       GIC 263 Edge      ppdu-end-interrupts-mac3
161:          0          0          0          0       GIC 262 Edge      ppdu-end-interrupts-mac2
162:          1          0          0      73383       GIC 261 Edge      ppdu-end-interrupts-mac1
163:          3          0          0          0       GIC 260 Edge      rxdma2host-monitor-status-ring-mac3
164:          0          0          0          0       GIC 256 Edge      rxdma2host-monitor-status-ring-mac2
165:          3          0          0          0       GIC 255 Edge      rxdma2host-monitor-status-ring-mac1
167:          0          0          0          0       GIC 215 Edge      host2rxdma-host-buf-ring-mac2
169:          0          0          0          0       GIC 211 Edge      rxdma2host-destination-ring-mac3
170:          0          0          0          0       GIC 210 Edge      rxdma2host-destination-ring-mac2
171:          0          0          0          0       GIC 209 Edge      rxdma2host-destination-ring-mac1
176:          0          0          0          0       GIC 191 Edge      wbm2host-tx-completions-ring3
182:          0          0          0          0       GIC 216 Edge      tsens_interrupt
183:          4          0          0          0       GIC  47 Edge      cpr3
185:        995          0      37160          0       GIC 107 Level     wlan_pci
186:          3          0          0          0  pmic_arb 3211277 Edge      spmi-vadc
187:          0          0          1          0     smp2p   1 Edge      error_ready_interrupt
188:          0          0          0          0     smp2p   0 Edge      err_fatal_interrupt
189:          0          0          0          0     smp2p   3 Edge      stop_ack_interrupt
190:          0          0          0          0       GIC 172 Edge      xhci-hcd:usb1
IPI0:     23349      61300      76753      59157       Rescheduling interrupts
IPI1:         6         11         11          9       Function call interrupts
IPI2:         0          0          0          0       CPU stop interrupts
IPI3:         0          0          0          0       Timer broadcast interrupts
IPI4:        62          0          0          2       IRQ work interrupts
IPI5:         0          0          0          0       CPU wakeup interrupts
Err:          0

dchard · 2021-11-17T11:58:48.274Z

Ahh... We need to wait a bit more then.

With the affinity patches, the loads are spread more evenly. Hopefully I can get my AX capable card within a week and we can do more proper tests to see where the limits are.

On the RX decap offload: are we sure that it actually works? mac80211 hooks are present? (refering to the discussion from last night)

Gruntruck · 2021-11-17T12:00:34.222Z

To actually see whether router running OpenWRT is CPU-limited we would need someone to run iperf3 between wired computer and WiFi client using a full fledged (at least) 2x2 802.11ax card. If you can push 1Gbit (which is limited by gbit ethernet) without maxing CPU then you are fine.

robimarko · 2021-11-17T12:01:40.016Z

I finally received a QCA6391 card, so I can test the PCI outside of AX9000 as well.
Thanks to @sumo for the donation.

I think that everything needed for RX decap is here as the mac80211 support for it was merged in early 2021, so it was in 5.12 as well.
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/net/mac80211?h=v5.15.2&id=80a915ec4427f0083829f7e6518ee9f21521ee1e

Gingernut · 2021-11-17T12:16:03.349Z

Just noticed that these two lines get inserted into /etc/sysctl.d/qca-nss-ecm.conf on every reboot even if they already exist?

net.bridge.bridge-nf-call-ip6tables=1
net.bridge.bridge-nf-call-iptables=1

Had maybe 10 duplicate entries.

dchard · 2021-11-17T12:30:13.767Z

Gingernut:

/etc/sysctl.d/qca-nss-ecm.conf

I can confirm that. I have way more than 10, and based on the last write timestamp, it happens at every sysupgrade or reboot. (likely the first one).

MOD: no, every reboot the extra two lines are added...

sumo · 2021-11-17T12:48:19.696Z

robimarko:

Thanks to @sumo for the donation.

Very welcome. Keep up the good work and let me know if I can be of any further help. Thanks!

dchard · 2021-11-17T13:37:49.194Z

@robimarko

A couple hours ago more patches arrived at Kalle's repo, and look what I have found:
ath11k: Fix ETSI regd with weather radar overlap

Looks exactly like hour issue. Other patches are also added which might worth take a look.

robimarko · 2021-11-17T13:43:31.964Z

My little bird already pointed me to that one a week ago, I applied it but I honestly didn't see a difference in the reported rules at all.
Feel free to give it a go, I might have messed something up.

Tomakava · 2021-11-17T14:02:23.780Z

I repeated same upload tests with affinity settings and speed is a lot higher and CPU now using all cores.

Screenshot_20211117-153534_Magic iPerf1079×2007 199 KB

I could try to test "RX decap" patch, but what exactly it should improve? Lower CPU usage in LAN -> WiFi case?

dchard · 2021-11-17T14:14:13.336Z

My joy was a bit too quick about that patch... It only fixes a driver warning, and not the wrong logic itself, which is likely not even on the driver's side. Non the less this line gives some info away:

If the firmware (or the BDF) is shipped with these rules

The task would be to understand the reg.c logic a bit better to make sure it cannot be fixed there. For example can someone explain to me that if a channel overlaps with the radar range, why is it changing its bandwidth instead of just applying the DFS an 600 sec CAC rule to the whole overlap band? Most ETSI countries are allowed to use 5470 - 5725 @ 160), yet ath11k splits it to two parts and creates a separate band in the middle where the radar range is. My point is, this might be reg.c, but my C skills are not that good to see it in the code. Someone with a bit more experience should take a look...

MOD:

Ok, I think I found where the extra split rules are created in reg.c:

/* Add max additional rules to accommodate weather radar band */
	if (reg_info->dfs_region == ATH11K_DFS_REG_ETSI)
		num_rules += 2;

and there are some extra elements in ath11k_reg_update_weather_radar_band to create the new rule boundaries.

Question is: why? I am not an expert on regulatory, but to my knowledge if a band or channel overlaps the ETSI radar band, the only limitation would be to do DFS with a CAC timeout of 600 seconds. As to my knowledge the ETSI radar band has not further power limitation other than what the countries limits for the designated band. So there is really no reason why reg.c splits the 5470 - 5725 @ 160 range to "lower", "radar" and "upper" chanels... It should just apply 600 seconds CAC to the whole band and that should be it.

castiel652 · 2021-11-17T14:43:05.192Z

Tomakava:

I could try to test "RX decap" patch, but what exactly it should improve? Lower CPU usage in LAN -> WiFi case?

Reduce CPU usage on WLAN -> router

mva · 2021-11-17T15:11:50.730Z

btw, @robimarko, do you build images on /releases page with defconfig, or do you added some additional options to .config?

robimarko · 2021-11-17T15:12:33.798Z

You can see whats inside here:

github.com

robimarko/openwrt/blob/IPQ807x-5.10-backports/.github/workflows/ipq807x.yaml#L18

    
      
          
          
steps:
              - name: Checkout
                uses: actions/checkout@v2
          
          
    - name: Update and install feeds
                run: |
                  ./scripts/feeds update -a
                  ./scripts/feeds install -a
          
          
    - name: Generate config for IPQ807x Generic
                run: |
                  echo "CONFIG_TARGET_ipq807x=y" > .config
                  echo "CONFIG_TARGET_ipq807x_generic=y\n" >> .config
                  echo "CONFIG_TARGET_MULTI_PROFILE=y\n" >> .config
                  echo "CONFIG_TARGET_ALL_PROFILES=y\n" >> .config
                  echo "CONFIG_TARGET_PER_DEVICE_ROOTFS=y\n" >> .config
                  echo "CONFIG_TARGET_ROOTFS_INITRAMFS=y\n" >> .config
                  echo "CONFIG_TARGET_INITRAMFS_COMPRESSION_ZSTD=y\n" >> .config
                  echo "CONFIG_PACKAGE_luci=y\n" >> .config
                  echo "CONFIG_PACKAGE_nss-firmware-ipq8074=y\n" >> .config

mva · 2021-11-17T15:14:43.008Z

lol. I intentionally looked in .github and failed to see anything like workflow yaml there.

(Although, looks like I looked it in master instead of -backports )

Thanks!