IPQ806x NSS Drivers

About that we have problems with patch author

2 Likes

yeah I saw that.
encountered a few problems with your repo but I managed to have a working build with drv and gmac.
Haven't looked into ECM yet.

It was very experimental but it worked... Extensive testing is required but I was a bit alone

1 Like

So which qsdk version would you suggest to work on?

Also I am still not sure how to deal with nss-firmware files.

1 Like

My suggestion at this point would be latest stable (what we target for ipq807x) and patch the qsdk to make it work with nss firmware 11.0 (only nss-core is needed to be patched)

That way we have more userbase.

2 Likes

@Ansuel you think this is related to the L2 freq issue?

Summary
<0>[561639.139282] Internal error: Oops - undefined instruction: 0 [#1] SMP ARM
<4>[561639.139308] Modules linked in: ecm iptable_nat ath10k_pci ath10k_core ath xt_state xt_nat xt_conntrack xt_REDIRECT xt_MASQUERADE xt_FLOWOFFLOAD wireguard nf_nat nf_flow_table_hw nf_flow_table nf_conntrack mac80211 libchacha20poly1305 ipt_REJECT ebtable_nat ebtable_filter ebtable_broute curve25519_neon cfg80211 xt_time xt_tcpudp xt_tcpmss xt_statistic xt_quota xt_pkttype xt_physdev xt_owner xt_multiport xt_mark xt_mac xt_limit xt_length xt_hl xt_ecn xt_dscp xt_comment xt_addrtype xt_TCPMSS xt_LOG xt_HL xt_DSCP xt_CLASSIFY ppp_async poly1305_arm nf_reject_ipv4 nf_log_ipv4 nf_defrag_ipv6 nf_defrag_ipv4 macvlan libcurve25519_generic l2tp_ppp iptable_mangle iptable_filter ipt_ECN ip_tables ebtables ebt_vlan ebt_stp ebt_redirect ebt_pkttype ebt_mark_m ebt_mark ebt_limit ebt_among ebt_802_3 crc_ccitt compat chacha_neon sch_tbf sch_ingress sch_htb sch_hfsc em_u32 cls_u32 cls_tcindex cls_route cls_matchall cls_fw cls_flow cls_basic act_skbedit act_mirred act_gact qca_nss_tunipip6 qca_nss_tun6rd
<4>[561639.139437]  qca_nss_ipsecmgr qca_nss_cfi_cryptoapi qca_nss_qdisc qca_nss_crypto qca_nss_vlan qca_nss_pppoe pppoe pppox ppp_generic slhc qca_nss_gre qca_nss_bridge_mgr ledtrig_usbport xt_set ip_set_list_set ip_set_hash_netportnet ip_set_hash_netport ip_set_hash_netnet ip_set_hash_netiface ip_set_hash_net ip_set_hash_mac ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_hash_ipport ip_set_hash_ipmark ip_set_hash_ip ip_set_bitmap_port ip_set_bitmap_ipmac ip_set_bitmap_ip ip_set nfnetlink nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 qca_mcs bonding ip6_gre ip_gre gre sit qca_nss_drv l2tp_netlink l2tp_core udp_tunnel ip6_udp_tunnel ipcomp6 xfrm6_tunnel esp6 ah6 xfrm4_tunnel ipcomp esp4 ah4 ipip ip6_tunnel qca_nss_gmac tunnel6 tunnel4 ip_tunnel tun qca_ssdk xfrm_user xfrm_ipcomp af_key xfrm_algo shortcut_fe_drv shortcut_fe_ipv6 shortcut_fe sha1_generic md5 echainiv des_generic libdes cbc authenc usb_storage leds_gpio xhci_plat_hcd
<4>[561639.210266]  xhci_pci xhci_hcd dwc3 dwc3_qcom ohci_platform ohci_hcd phy_qcom_ipq806x_usb ahci fsl_mph_dr_of ehci_platform ehci_fsl sd_mod ahci_platform libahci_platform libahci libata scsi_mod ehci_hcd gpio_button_hotplug ext4 mbcache jbd2 crc32c_generic
<4>[561639.320122] CPU: 1 PID: 2488 Comm: sugov:1 Not tainted 5.4.211 #0
<4>[561639.342350] Hardware name: Generic DT based system
<4>[561639.348351] PC is at regulator_check_voltage+0x8c/0x128
<4>[561639.353117] LR is at regulator_set_voltage_unlocked+0x98/0x134
<4>[561639.358671] pc : [<c06403d8>]    lr : [<c0642ce4>]    psr: 20000113
<4>[561639.364313] sp : d9653e28  ip : 0010c8e0  fp : dcea8370
<4>[561639.370910] r10: dcea8200  r9 : dcea64c0  r8 : 0010c8e0
<4>[561639.376204] r7 : 3b9aca00  r6 : 00000000  r5 : dccb0e00  r4 : dcea6500
<4>[561639.381500] r3 : 00118c30  r2 : d9653e40  r1 : d9653e44  r0 : dccb0e00
<4>[561639.387839] Flags: nzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
<4>[561639.394437] Control: 10c5787d  Table: 5a82406a  DAC: 00000051
<0>[561639.401728] Process sugov:1 (pid: 2488, stack limit = 0x54b2b4a1)
<0>[561639.407543] Stack: (0xd9653e28 to 0xd9654000)
<0>[561639.413714] 3e20:                   dcea6500 dccb0e00 00000000 3b9aca00 0010c8e0 c0642ce4
<0>[561639.418149] 3e40: 0010c8e0 00118c30 dcea6500 00118c30 0010c8e0 c0643004 d93ee880 000767c0
<0>[561639.426395] 3e60: 00000002 00000000 dcea8340 3b9aca00 16e36000 c073c5ac dcea6500 00118c30
<0>[561639.434642] 3e80: d93ebc80 dcc7b400 3b9aca00 dcea8340 3b9aca00 dcea6600 dcea6600 00000010
<0>[561639.442888] 3ea0: dd52ca10 c0731518 dcc7b400 c07303a4 dcea6634 00000001 dcc7b400 3b9aca00
<0>[561639.451135] 3ec0: dcea6608 dcea6600 3b9aca00 00000001 c0c04f28 00000001 c0c673cc dcc7b000
<0>[561639.459382] 3ee0: 00000000 c073c014 d9653f20 dcea3f80 53724e00 00155cc0 d8d62018 dcc7b000
<0>[561639.467629] 3f00: 00000000 c0c673a8 00000000 00000004 00155cc0 00000000 d96e46dc c0735c68
<0>[561639.475874] 3f20: dcc7b000 000f4240 00155cc0 000000a1 d87f693c d87f6950 00000000 00155cc0
<0>[561639.484121] 3f40: 00000001 d87f696c d8819d7c c0367030 d87f693c d87f6964 ffffe000 d87f6968
<0>[561639.492368] 3f60: 00000001 c033dc0c d96e46c0 d8160e80 00000000 d9652000 c033db7c d87f6964
<0>[561639.500614] 3f80: d8819d7c c033eb3c 00000000 d8160e80 c033e9dc 00000000 00000000 00000000
<0>[561639.508861] 3fa0: 00000000 00000000 00000000 c03010e8 00000000 00000000 00000000 00000000
<0>[561639.517106] 3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<0>[561639.525352] 3fe0: 00000000 00000000 00000000 00000000 00000013 00000000 00000000 00000000
<4>[561639.533598] [<c06403d8>] (regulator_check_voltage) from [<c0642ce4>] (regulator_set_voltage_unlocked+0x98/0x134)
<4>[561639.541844] [<c0642ce4>] (regulator_set_voltage_unlocked) from [<c0643004>] (regulator_set_voltage+0x3c/0x68)
<4>[561639.552177] [<c0643004>] (regulator_set_voltage) from [<c073c5ac>] (krait_cache_set_opp+0xc8/0x150)
<4>[561639.562073] [<c073c5ac>] (krait_cache_set_opp) from [<c0731518>] (dev_pm_opp_set_rate+0x158/0x49c)
<4>[561639.571354] [<c0731518>] (dev_pm_opp_set_rate) from [<c073c014>] (set_target+0x164/0x1ec)
<4>[561639.580118] [<c073c014>] (set_target) from [<c0735c68>] (__cpufreq_driver_target+0x1a0/0x568)
<4>[561639.588458] [<c0735c68>] (__cpufreq_driver_target) from [<c0367030>] (sugov_work+0x54/0x70)
<4>[561639.597051] [<c0367030>] (sugov_work) from [<c033dc0c>] (kthread_worker_fn+0x90/0x174)
<4>[561639.605640] [<c033dc0c>] (kthread_worker_fn) from [<c033eb3c>] (kthread+0x160/0x164)
<4>[561639.613367] [<c033eb3c>] (kthread) from [<c03010e8>] (ret_from_fork+0x14/0x2c)
<4>[561639.621345] Exception stack(0xd9653fb0 to 0xd9653ff8)
<4>[561639.628466] 3fa0:                                     00000000 00000000 00000000 00000000
<4>[561639.633685] 3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<4>[561639.641929] 3fe0: 00000000 00000000 00000000 00000000 00000013 00000000
<0>[561639.650174] Code: e1cd60d8 e59d8010 e28dd014 e49df004 (e7f001f2) 
<4>[561639.657023] ---[ end trace 56f064d345e8cfb3 ]---
<0>[561639.675386] Kernel panic - not syncing: Fatal exception
<2>[561639.675414] CPU0: stopping
<4>[561639.679751] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G      D           5.4.211 #0
<4>[561639.682272] Hardware name: Generic DT based system
<4>[561639.689917] [<c030fad4>] (unwind_backtrace) from [<c030b9f0>] (show_stack+0x14/0x20)
<4>[561639.694605] [<c030b9f0>] (show_stack) from [<c09187d8>] (dump_stack+0x94/0xa8)
<4>[561639.702587] [<c09187d8>] (dump_stack) from [<c030ed00>] (handle_IPI+0x184/0x1b8)
<4>[561639.709715] [<c030ed00>] (handle_IPI) from [<c05e9c80>] (gic_handle_irq+0xb4/0xb8)
<4>[561639.717348] [<c05e9c80>] (gic_handle_irq) from [<c0301b0c>] (__irq_svc+0x6c/0x90)
<4>[561639.724808] Exception stack(0xc0c01ee0 to 0xc0c01f28)
<4>[561639.732453] 1ee0: 00000000 0001fece 1ce4c000 dd98fb00 dcc43000 00000000 dd98eeb0 0001fece
<4>[561639.737578] 1f00: 0001fece 00000000 ed583100 ed4c8e40 00000015 c0c01f30 c073cbf8 c073cbfc
<4>[561639.745818] 1f20: 80000013 ffffffff
<4>[561639.754061] [<c0301b0c>] (__irq_svc) from [<c073cbfc>] (cpuidle_enter_state+0x94/0x498)
<4>[561639.757799] [<c073cbfc>] (cpuidle_enter_state) from [<c073d044>] (cpuidle_enter+0x30/0x4c)
<4>[561639.765873] [<c073d044>] (cpuidle_enter) from [<c034c20c>] (do_idle+0x1d8/0x240)
<4>[561639.773943] [<c034c20c>] (do_idle) from [<c034c51c>] (cpu_startup_entry+0x1c/0x20)
<4>[561639.781584] [<c034c51c>] (cpu_startup_entry) from [<c0b00fb4>] (start_kernel+0x4c8/0x4d8)

Yep every kernel panic with undefined instruction/strange memory access that are not null or an error are thing related to the cpu going on a acid trip

I built it by including the qca-mcs package.
Can you tell me how to enable multicast snooping with nss?

Having the qca-mcs module loaded is not enough. Your need to also patch the IGMP/MLD mechanism you are using for multicast processing. I patched igmpproxy to trigger qca-mcs to accelerate multicast traffic for my ISP IPTV service.

It’s working somewhat and needs to be enhanced. As I need to use the router for actual use I abandoned the project tho and use normal Linux kernel multicast processing. It’s about 5% CPU saved, so my lazy bone took over and I abandoned the project. Haha.

If you are interested to continue where I left off, here’s the changes to igmpproxy:

HTH.

3 Likes

I changed the version of igmpproxy from 0.2.1 to 0.4 and built it with some modifications of your patch but openwrt failed to allow multicast traffic from iptv.

Did it work before?

My patch to igmpproxy basically configures the NSS drivers. It will not interfere with how igmpproxy works. Looks like you need to figure out how your IPTV service works with igmpproxy first.

igmpproxy works fine on non-patch builds.
I uncommented /etc/config/igmpproxy in a build with your patch and set it up as usual, and it didn't work.

config igmpproxy
option quickleave 1
# option verbose [0-3](none, minimal[default], more, maximum)

config phyint
option network wan
option zone wan
option direction upstream
list altnet 0.0.0.0/0

config phyint
option network lan
option zone lan
option direction downstream

# This section configures the QCA MCS support
# Uncomment to enable QCA NSS ECM multicast acceleration.
#
# 'lan' should be the LAN bridge device
# 'brport' should be the bridge port(s)
#
# Up to 16 bridges with up to 16 bridge ports(per bridge) can be configured.
config phyint
option network lan
option zone lan
option direction qca-mcs-bridge
list brport eth1.1

Could it be because a lot of packages have been updated in the meantime, including openwrt and drivers?

Possible that there’s conflicts with packages. Do you see any errors logged?

Where can I see it? system log? kernel log?

Igmpproxy logs to syslog.

Before removing the comment, I was able to see the usual igmpproxy logs.

However, if I change the settings in /etc/config/igmpproxy to the following, I see the following log and no multicast traffic is processed.

config igmpproxy
        option quickleave 1
#       option verbose [0-3](none, minimal[default], more, maximum)

config phyint
        option network wan
        option zone wan
        option direction upstream
        list altnet 0.0.0.0/0

config phyint
        option network lan
        option zone lan
        option direction downstream

# This section configures the QCA MCS support
# Uncomment to enable QCA NSS ECM multicast acceleration.
#
# 'lan' should be the LAN bridge device
# 'brport' should be the bridge port(s)
#
# Up to 16 bridges with up to 16 bridge ports(per bridge) can be configured.
config phyint
       option network lan
       option zone lan
       option direction qca-mcs-bridge
       list brport eth1.1

Wed Jul 19 01:29:46 2023 kern.info kernel: [ 995.303211] mc_detach: disabled snooping on br-lan.
Wed Jul 19 01:29:46 2023 kern.info kernel: [ 995.311292] mc_attach: enabled snooping on br-lan.

Is your router configured with different vlan for the bridge?

The config list brport eth1.1 means the LAN bridge is configured to use eth1 on VLAN 1. You will need to adjust the config if you have changed the LAN bridge VLAN.

Edit: I would add that if you want to use multicast streams in your network, you should also configure the bridge to turn on IGMP snooping, or you may flood your switch ports with multicast traffic, which may also flood the WLAN interfaces.

1 Like

I enabled and tested the options below.

uci set network.@device[0].igmp_snooping='1'
uci set network.@device[1].igmp_snooping='1'

It has VLAN named eth0.2 and eth1.1.

I usually use Luci to configure. I find using uci command too finicky as there are sometimes dependencies that I may missed out on.

In any case, I can’t really tell what’s the issue you’re facing without actually testing out the build you’re using. Most likely the qca-mcs driver you’re using have changed the netlink APIs from the one I used.

1 Like

Today I have builded OpenWRT v23 firmware for Xiaomi R3D with built-in NSS driver.

Sources:

xiaomi_r3d.dts based on netgear_r7500v2.dts

Speed of WiFi 5G (80Mhz):
w/o NSS: 396 / 380 Mbps
with NSS: 680 / 363 Mbps

What else should be checked to improve performance?
How to check firmware stability?
How to check for memory leaks?

1 Like