I looked at it, but seen no sign of 11.4.1 anywhere on codeaurora. Where can I find it?
@Gingernut most I was referring to the modifications 11.5 will bring, so Robi might be right that we need to sit this out until 11.5 is released.
2 Likes
Sometimes they have a branch for it, otherwise it's part of the 11.4 branches under all of the components.
1 Like
As much as I can see, both the DP and the DRV part mostly has interesting parts as of the 11.5 branch. But maybe I am missing something. The EDMAv2 section also looks interesting, that is also 11.5 as well.
MOD: btw I am using this NSS FW link for two weeks now, one subversion newer compared to what we have. You might want to apply it, its a bugfix minor release. Works as expected on my end.
MOD: the TIp project page link has a large amount of mods for the NSS DRV, DP etc. as part of 11.4, maybe we can pull those and lets see what happens.
11.5 has a ton of new features, but there is no FW public yet.
TIP has some ath11k hacks, but other than that they don't really have anything special.
Its probably also worth mentioning the kernel patches to add the nss hooks was the minimal to get pppoe and nat acceleration working. High chance we are missing other changes.
(Unless things have changed) we Ported the bare minimum kernel parts to keep the patches somewhat portable.
finally i was able to flash openwrt
however my eth & wlan still broken, but didnt crash/panics
could you please check this log @robimarko ?
[ 6.720406] kmodloader: loading kernel modules from /etc/modules.d/*
[ 6.727756] urngd: v1.0.2 started.
[ 6.902655] ssdk_switch_device_num_init[1160]:INFO:ess-switch dts node number: 1
[ 6.902747] ssdk_dt_get_switch_node[971]:INFO:ess-switch DT exist!
[ 6.909136] ssdk_dt_parse_access_mode[857]:INFO:switch_access_mode: local bus
[ 6.915113] ssdk_dt_parse_access_mode[870]:INFO:switchreg_base_addr: 0x3a000000
[ 6.922316] ssdk_dt_parse_access_mode[871]:INFO:switchreg_size: 0x1000000
[ 6.929443] ssdk_dt_parse_mac_mode[295]:INFO:mac mode = 0x0
[ 6.936378] ssdk_dt_parse_mac_mode[304]:INFO:mac mode1 = 0xff
[ 6.941759] ssdk_dt_parse_mac_mode[313]:INFO:mac mode2 = 0xff
[ 6.947709] ssdk_dt_parse_uniphy[332]:INFO:ess-uniphy DT exist!
[ 6.953589] ssdk_dt_parse_intf_mac[800]:INFO:dp3 MAC 9c:9d:7e:1c:03:04
[ 6.959157] ssdk_dt_parse_intf_mac[800]:INFO:dp4 MAC 9c:9d:7e:1c:03:04
[ 6.965756] ssdk_dt_parse_intf_mac[800]:INFO:dp5 MAC 9c:9d:7e:1c:03:04
[ 6.972274] ssdk_plat_init start
[ 7.027329] ------------[ cut here ]------------
[ 7.027351] gcc_nss_port1_rx_clk status stuck at 'off'
[ 7.027386] WARNING: CPU: 0 PID: 927 at clk_branch_wait+0x130/0x140
[ 7.035952] Modules linked in: qca_ssdk(+) seqiv jitterentropy_rng drbg michael_mic hmac cmac leds_gpio xhci_plat_hcd xhci_pci xhci_hcd dwc3 dwc3_qcom gpio_button_hotplug
[ 7.042139] CPU: 0 PID: 927 Comm: kmodloader Not tainted 5.10.82 #0
[ 7.057393] Hardware name: Redmi AX6 (DT)
[ 7.063556] pstate: 60400085 (nZCv daIf +PAN -UAO -TCO BTYPE=--)
[ 7.067725] pc : clk_branch_wait+0x130/0x140
[ 7.073798] lr : clk_branch_wait+0x130/0x140
[ 7.078049] sp : ffffffc0117e3720
[ 7.082301] x29: ffffffc0117e3720 x28: ffffffc00879fc24
[ 7.085516] x27: 0000000000000001 x26: ffffffc00878f000
[ 7.090899] x25: ffffffc0088171d2 x24: ffffffc008817138
[ 7.096192] x23: ffffffc01084fc78 x22: 0000000000000001
[ 7.101488] x21: ffffffc0103d67a0 x20: 0000000000000000
[ 7.106784] x19: ffffffc010a92910 x18: 00000000000000d4
[ 7.112078] x17: 0000000000000000 x16: 0000000000000000
[ 7.117373] x15: ffffffc010a07d48 x14: 000000000000027c
[ 7.122670] x13: 00000000000000d4 x12: ffffffc0117e33f8
[ 7.127964] x11: fffffffffffe31e8 x10: fffffffffffe31b0
[ 7.133259] x9 : ffffffc010a5fd48 x8 : 00000000000013e0
[ 7.138555] x7 : ffffffc010a07d48 x6 : 00000000ffffefff
[ 7.143849] x5 : ffffffc010a5fd48 x4 : 0000000000000000
[ 7.149144] x3 : 0000000000000000 x2 : 0000000000000000
[ 7.154440] x1 : ffffffc010a07ca8 x0 : 000000000000002a
[ 7.159736] Call trace:
[ 7.165027] clk_branch_wait+0x130/0x140
[ 7.167199] clk_branch2_enable+0x34/0x44
[ 7.171367] clk_core_enable+0x6c/0xc0
[ 7.175273] clk_enable+0x2c/0x60
[ 7.179423] hppe_host_flush_common+0x140/0x160 [qca_ssdk]
[ 7.182716] ssdk_uniphy_clock_enable+0x88/0x148 [qca_ssdk]
[ 7.188044] ssdk_uniphy_port5_clock_source_set+0x4e0/0x730 [qca_ssdk]
[ 7.193503] ssdk_gcc_clock_init+0xe4/0x200 [qca_ssdk]
[ 7.200097] ssdk_plat_init+0x33c/0x3d0 [qca_ssdk]
[ 7.205220] init_module+0x228/0x1000 [qca_ssdk]
[ 7.209652] do_one_initcall+0x50/0x1b0
[ 7.214423] do_init_module+0x54/0x1f0
[ 7.217979] load_module+0x1c78/0x2310
[ 7.221799] __do_sys_init_module+0x170/0x270
[ 7.225533] __arm64_sys_init_module+0x20/0x30
[ 7.229962] el0_svc_common.constprop.0+0x88/0x190
[ 7.234302] do_el0_svc+0x74/0x94
[ 7.239076] el0_svc+0x14/0x20
[ 7.242459] el0_sync_handler+0xa8/0x130
[ 7.245411] el0_sync+0x184/0x1c0
[ 7.249489] ---[ end trace ca0015cb0550ea90 ]---
[ 7.252817] ssdk_uniphy_clock_enable[162]:ERROR:clock enable fail!
[ 7.257498] ssdk_gcc_clock_init[1033]:INFO:SSDK gcc clock init successfully!
[ 7.264409] HPPE initializing...
[ 7.270905] malibu_phy_api_ops_init[2848]:INFO:qca probe malibu phy driver succeeded!
[ 7.279181] regi_init[3567]:INFO:Initializing HPPE!!
[ 7.505541] ssdk_ppe_reset_init[1265]:INFO:ppe reset successfully!
[ 7.508911] qca_hppe_tdm_hw_init[684]:INFO:tdm setup num=96
[ 7.510947] qca_hppe_portctrl_hw_init[110]:INFO:Hawkeye PPE port initializing
[ 7.731755] ssdk_switch_register[1718]:INFO:Chip version 0x1500
[ 7.731786] qca_link_polling_select[1315]:INFO:link-polling-required node does not exist
[ 7.736481] ssdk_switch_register[1744]:INFO:polling is selected
[ 7.744816] regi_init[3571]:INFO:Initializing HPPE Done!!
[ 7.750523] regi_init[3631]:INFO:qca-ssdk module init succeeded!
[ 7.768686] EDMA ver 1 hw init
[ 7.768935] EDMA HW Reset completed succesfully
[ 7.770694] Num rings - TxDesc:1 (23-23) TxCmpl:1 (7-7)
[ 7.775064] RxDesc:1 (15-15) RxFill:1 (7-7)
[ 7.785406] **********************************************************
[ 7.785437] * NSS Data Plane driver
[ 7.791096] **********************************************************
[ 7.819137] qca-nss 39000000.nss: Direct firmware load for qca-nss0.bin failed with error -2
[ 7.819179] qca-nss 39000000.nss: Falling back to sysfs fallback for: qca-nss0.bin
/lib/firmware/qca-nss0.bin 33
[ 8.037842] nss_driver - fw of size 833304 bytes copied to load addr: 40000000, nss_id : 0
[ 8.040213] Supported Frequencies -
[ 8.040221] 187.2 MHz
[ 8.045026] 748.8 MHz
[ 8.048831] 1.4976 GHz
[ 8.050988]
[ 8.055757] ffffffc008955900: set sdma ffffff8002ab5200
[ 8.057529] ffffffc008955900: meminfo init succeed
[ 8.081725] qca-nss 39400000.nss: Direct firmware load for qca-nss1.bin failed with error -2
[ 8.081765] qca-nss 39400000.nss: Falling back to sysfs fallback for: qca-nss1.bin
[ 8.101969] node size 2 # items 4
[ 8.102000] memory: 40000000 536870912 (avl 436760576) items 4 active_cores 2
[ 8.104279] addr/size storage words 2 2 # words 4 in DTS, ddr size 1000000
[ 8.111433] ffffffc008955900: nss core 0 booted successfully
/lib/firmware/qca-nss1.bin 33
[ 8.167747] nss_driver - fw of size 292296 bytes copied to load addr: 40800000, nss_id : 1
[ 8.168743] Supported Frequencies -
[ 8.168749] 187.2 MHz
[ 8.174965] 748.8 MHz
[ 8.178738] 1.4976 GHz
[ 8.180885]
[ 8.185668] ffffffc00895d140: set sdma ffffff8002b1aa00
[ 8.187389] ffffffc00895d140: meminfo init succeed
[ 8.192444] debugfs: Directory 'dynamic_if' with parent 'stats' already present!
[ 8.197143] debugfs: File 'n2h' in directory 'strings' already present!
[ 8.204688] debugfs: File 'drv' in directory 'strings' already present!
[ 8.215494] node size 2 # items 4
[ 8.217635] memory: 40000000 536870912 (avl 436994048) items 4 active_cores 2
[ 8.221105] addr/size storage words 2 2 # words 4 in DTS, ddr size 1000000
[ 8.228232] ffffffc00895d140: nss core 1 booted successfully
[ 8.239223] PPP generic driver version 2.4.2
[ 8.241628] NET: Registered protocol family 24
[ 8.248421] Loading modules backported from Linux version v5.15-rc6-0-g519d81956ee2
[ 8.249300] Backport generated by backports.git v5.15-rc6-1-0-gd44432d6
[ 8.270638] xt_time: kernel timezone is -0000
[ 8.311186] ath11k c000000.wifi: ipq8074 hw2.0
[ 8.311452] remoteproc remoteproc0: powering up cd00000.q6v5_wcss
[ 8.335825] remoteproc remoteproc0: Booting fw image IPQ8074/q6_fw.mdt, size 668
[ 9.633424] remoteproc remoteproc0: remote processor cd00000.q6v5_wcss is now up
[ 9.635944] kmodloader: done loading kernel modules from /etc/modules.d/*
[ 13.669295] ECM init
[ 13.669358] ECM database jhash random seed: 0x54e53765
[ 13.671915] ECM init complete
[ 14.528860] br-lan: port 1(eth1) entered blocking state
[ 14.528896] br-lan: port 1(eth1) entered disabled state
[ 14.533602] device eth1 entered promiscuous mode
[ 14.548108] br-lan: port 2(eth2) entered blocking state
[ 14.548146] br-lan: port 2(eth2) entered disabled state
[ 14.553053] device eth2 entered promiscuous mode
[ 14.561769] br-lan: port 3(eth3) entered blocking state
[ 14.562277] br-lan: port 3(eth3) entered disabled state
[ 14.568062] device eth3 entered promiscuous mode
[ 14.578185] nss-dp 3a001200.dp2 eth0: PHY Link up speed: 10
[ 15.553607] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[ 31.713561] pmd9655_ldo11: disabling
device ax6
openwrt build 456cbdf
base fw 1.0.16
@dchard @robimarko @Ansuel @Apache14 could the ipq807x platform also benefit from this addition by @KONG as we are using some of the same kernel modules?
Robimarko has said it several times he doesn't want to have that much QCA hacky stuff.
and this won't help making it into upstream OpenWrt
2 Likes
I'm not an expert, but KONG's patch seems interesting - thanks for sharing @Gingernut
I do understand frustrations because the state of Qualcomm code, and while the benefits for IPQ806x will be probably much bigger compared to IPQ807x, it still might be worth to port this patch.
The patch provided by @KONG is meant to "accelerate" WiFi traffic in the client -> AP direction, and it requires the use of the NSS ECM module, i.e. the use of qca-nss-drv, qca-nss-gmac and the NSS firmware. It's main purpose is to bypass netfilter processing (i.e. the fast path) for established connection.
In other words, it's probably not useful for the AX3600 ipq807x project unless the NSS cores are used with the ECM module for network acceleration.
To speed up WiFi for the ipq807x project, it's better to try the patch ath11k to support tx encap offload (if possible), similar to the patch that was done for the ath10k.
Both TX encap and RX decap offload are already supported in the backport driver.
3 Likes
the """offload""" for ipq806x is very basic and really not very efficient... ipq807x supports real wifi offload where the nss core actually handle wifi packet and not only the packets
Has anyone tried to realize guest wifi ? There are many solutions in the internet. I managed to make a separate IP - Range and the wireless device gets an IP Address from the dhcp. But it is not possible, to have an interent connection. Do I habe to create a NAT Rule in the Firewall (former manuals do this, in recent manuals there aren't any NAT Rules) or isn't it possible with the prebulid to make guest WLAN with NAT ?
And yet, when you do 160MHz AX you will see massive CPU load (kernel) across all cores. So reducing that with whatever technique would be useful.
On the VLAN/L2 issues: as much as I understood when Robi explained it to me, a lot of offload actually happens in the PPE engine which is not part of the NSS, but more like the ethernet driver, but maybe I am wrong. Not that we have a proper ethernet driver at the moment...
1 Like
The problem with Linux is that netfilter will process all packets thru the same rules, even when there’re no changes applied. This exact a high toll on the CPU, especially for less powerful router SoCs. That’s why flow-offload alternatives (NSS, Linux flow-offload, QCA SFE, etc) bypasses netfilter for established connections to work-around it, thereby achieving acceleration. They higher the bandwidth the worst it’ll be for router SoCs. This is the main reason I tried to enable the NSS cores for ipq806x.
could someone create full partition backup for AX6?
planned to flash full partitions, just to make sure its hardware fault or corrupted nand.
I kind of doubt it's the NAND, it's failing to enable the clock for the port and thus failing.
I have not seen this so far.
@dchard The PPE is an HW IP block, it is just used in a way that the NSS FW will make calls to SSDK to configure it and that is how they are offloading the bridge, VLAN, NAT, and stuff like that.
I am sure it can handle some L3 as well, but without a datasheet good luck in ever getting it to work outside of the NSS ecosystem.
@Gingernut In theory it can use the same packages, but the issue is that the NSS FW for IPQ806x that is available is way older.
Thanks for clearing that up. So for PPE to work we do need NSS, if I understand it correctly.
Can you tell us a bit more about what is the fundamental difference between Qualcomm's NSS, and Mediatek's HW acceleration? I wonder how can that made it upstream while NSS seems so out of reach from upstreaming point of view.
Actually not, if one had a datasheet it could work without NSS by using the existing flow offloading in the kernel as it has no FW, it's an HW MMIO block.
NSS is a name that Qualcomm uses for everything when it comes to networking offloading, so it encompasses the HNAT inside of the switch, PPE if the HW has it, the UBI32 NSS cores themselves, EIP crypto block with its own FW that is part of the second NSS core and WLAN drivers.
MediaTek's offloading made upstream as it's just the PPE itself and somebody got ahold of the datasheet, so there is no FW involved.
So if one had the IPQ807x PPE datasheet that could be utilized on its own after you have a ethernet driver and a DSA based switch driver.
Stock ssdk probe was successfull @robimarko.
however, qca_ol.ko (wlan offload?) cause kernel panics.
stock ssdk:
[ 11.988849] ssdk_switch_device_num_init[998]:INFO:ess-switch dts node number: 1
[ 11.992555] ssdk_dt_get_switch_node[836]:INFO:ess-switch DT exist!
[ 11.999472] ssdk_dt_parse_access_mode[737]:INFO:switch_access_mode: local bus
[ 12.005623] ssdk_dt_parse_access_mode[750]:INFO:switchreg_base_addr: 0x3a000000
[ 12.012748] ssdk_dt_parse_access_mode[751]:INFO:switchreg_size: 0x1000000
[ 12.020038] ssdk_dt_parse_mac_mode[242]:INFO:mac mode = 0x0
[ 12.026454] ssdk_dt_parse_mac_mode[251]:INFO:mac mode1 = 0xff
[ 12.032016] ssdk_dt_parse_mac_mode[260]:INFO:mac mode2 = 0xff
[ 12.037767] ssdk_dt_parse_uniphy[279]:INFO:ess-uniphy DT exist!
[ 12.043579] ssdk_dt_parse_intf_mac[683]:INFO:dp1 MAC 9c:9d:7e:1c:03:04
[ 12.049401] ssdk_dt_parse_intf_mac[683]:INFO:dp2 MAC 28:d1:27:f5:a6:1b
[ 12.055906] ssdk_dt_parse_intf_mac[683]:INFO:dp3 MAC 9c:9d:7e:1c:03:04
[ 12.062422] ssdk_dt_parse_intf_mac[683]:INFO:dp4 MAC 9c:9d:7e:1c:03:04
[ 12.068933] ssdk_dt_parse_intf_mac[683]:INFO:dp5 MAC 9c:9d:7e:1c:03:04
[ 12.075455] ssdk_plat_init start
[ 12.160746] ssdk_ppe_clock_init[464]:INFO:ppe and uniphy clock init successfully!
[ 12.163908] ssdk_probe[1884]:INFO:ess_rst doesn't exist!
[ 12.171814] HPPE initializing...
[ 12.172484] malibu_phy_api_ops_init[2851]:INFO:qca probe malibu phy driver succeeded!
[ 12.183541] regi_init[3411]:INFO:Initializing HPPE!!
[ 12.398634] ssdk_ppe_reset_init[738]:INFO:ppe reset successfully!
[ 12.405945] qca_hppe_tdm_hw_init[737]:INFO:tdm setup num=96
[ 12.409228] qca_hppe_portctrl_hw_init[165]:INFO:Hawkeye PPE port initializing
[ 12.624743] ssdk_switch_register[1623]:INFO:Chip version 0x1500
[ 12.627726] qca_link_polling_select[1285]:INFO:link-polling-required node does not exist
[ 12.633963] ssdk_switch_register[1648]:INFO:polling is selected
[ 12.641694] regi_init[3415]:INFO:Initializing HPPE Done!!
[ 12.646620] regi_init[3459]:INFO:qca-ssdk module init succeeded!
crash:
[ 51.401660] wlan: [3771:I:ANY] ol_ath_ahb_probe: 504: Error: QMI FW ready timeout 15 seconds
[ 51.401693] qmi_history_index [127]
[ 51.409167] Assertion failed! 0:ol_ath_ahb_probe /home/jenkins/romdaily_new_openwrt/system/build_dir/target-aarch64-openwrt-linux_musl/linux-ipq807x_64_MiWiFi/qca-wifi-miwifi_git-unified-profile/qca-wifi-miwifi_git/os/linux/../../qca_ol/../offload/os/linux/ath_ahb.c:506
[ 51.414095] CPU: 3 PID: 3771 Comm: insmod Tainted: P 4.4.60 #0
[ 51.436328] Hardware name: Qualcomm Technologies, Inc. IPQ807x/AP-AC04 (DT)
[ 51.443534] Call trace:
[ 51.450303] [<ffffffc000087b78>] dump_backtrace+0x0/0x128
[ 51.452734] [<ffffffc000087cb4>] show_stack+0x14/0x1c
[ 51.458291] [<ffffffc00027c1c0>] dump_stack+0x94/0xb4
[ 51.463676] [<ffffffbffd2f3300>] ol_ath_ahb_probe+0xd0/0x508 [qca_ol]
[ 51.468677] [<ffffffbffd2f38e4>] ol_ath_ahb_ssr+0x1ac/0x6f8 [qca_ol]
[ 51.475092] [<ffffffbffd2f4354>] ol_ath_pld_probe+0x1c/0x38 [qca_ol]
[ 51.481510] [<ffffffbffd2b8bb4>] pld_pcie_device_self_recovery+0x23c/0x268 [qca_ol]
[ 51.487548] [<ffffffc0003a4b1c>] cnss_qca8074_notifier_nb+0x28/0x7c
[ 51.494925] [<ffffffc0000abf30>] notifier_call_chain+0x50/0x84
[ 51.501173] [<ffffffc0000ac410>] __srcu_notifier_call_chain+0x80/0xac
[ 51.507076] [<ffffffc0000ac450>] srcu_notifier_call_chain+0x14/0x1c
[ 51.513588] [<ffffffc000305a40>] subsys_notif_queue_notification+0x38/0x4c
[ 51.519664] [<ffffffc00030706c>] notify_each_subsys_device.isra.6+0xa4/0xc0
[ 51.526608] [<ffffffc000307b78>] __subsystem_get+0x148/0x1a0
[ 51.533463] [<ffffffc000307be0>] subsystem_get+0x10/0x18
[ 51.539367] [<ffffffc0003a6ad8>] cnss_wlan_register_driver+0x9c/0x158
[ 51.544972] [<ffffffbffd2b8c20>] pld_ahb_register_driver+0x10/0x20 [qca_ol]
[ 51.551303] [<ffffffbffd2b7504>] pld_register_driver+0x8c/0x148 [qca_ol]
[ 51.558078] [<ffffffbffd2f4310>] pld_module_init+0x10/0x20 [qca_ol]
[ 51.564876] [<ffffffbffd494468>] init_ath_ahb_3_0+0x50/0xf0 [wifi_3_0]
[ 51.570861] [<ffffffbffd4d8014>] init_module+0x14/0x168 [wifi_3_0]
[ 51.577303] [<ffffffc000082218>] do_one_initcall+0x19c/0x1b8
[ 51.583465] [<ffffffc000108f28>] do_init_module+0x58/0x19c
[ 51.589280] [<ffffffc0000ef2a0>] load_module+0x1660/0x1adc
[ 51.594574] [<ffffffc0000ef85c>] SyS_init_module+0x140/0x16c
[ 51.600043] [<ffffffc0000855b0>] el0_svc_naked+0x24/0x28
[ 51.605891] QDF BUG in ol_ath_ahb_probe Line 506: Failed assertion '0'