Qualcommax NSS Build

Yup it's already enabled during the compilation process

CONFIG_NSS_DRV_PPPOE_ENABLE=y
CONFIG_NSS_DRV_PPTP_ENABLE=y
# CONFIG_NSS_DRV_PVXLAN_ENABLE is not set
# CONFIG_NSS_DRV_QRFS_ENABLE is not set
# CONFIG_NSS_DRV_QVPN_ENABLE is not set
CONFIG_NSS_DRV_RMNET_ENABLE=y
# CONFIG_NSS_DRV_SHAPER_ENABLE is not set
# CONFIG_NSS_DRV_SJACK_ENABLE is not set
# CONFIG_NSS_DRV_TLS_ENABLE is not set
Applying /home/lin/ax6-6.x/package/kernel/mac80211/patches/nss/subsys/971-mac80211-fix-field-span-warning.patch using plaintext: 
patching file include/linux/ieee80211.h
Hunk #1 FAILED at 961.
1 out of 1 hunk FAILED -- saving rejects to file include/linux/ieee80211.h.rej
Patch failed!  Please fix /home/lin/ax6-6.x/package/kernel/mac80211/patches/nss/subsys/971-mac80211-fix-field-span-warning.patch!

Please refrain from posting issues related to local branch merges here. The issue is only when you pull from main on top of my branch.

I specifically don't push my rebase until I can verify a build goes through successfully. As a result, I'm not in realtime sync.

2 Likes

I had seen this in prior builds, mostly in 6.6.30-32. These are always difficult to pin down. 6.6.33 has been the most stable for me on mx5300, and dl-wrx36. Both are on 1GB profile.

There were some changes in the subsys patches for mac80211 upstream, but I reverted them in a patch. Have you tried with 6.6.34 from today?

I had originally thought rmnet_nss_callbacks was coming from one of the QCA nss modules, but I don't see it anywhere in the sources. Are you applying a custom patch for RMNET? I searched my entire Qualcomm QSDK folder and the only mention I see of it is from kernel 5.4.

❯ rg rmnet_nss_callbacks
qsdk-5.4/qca/src/datarmnet/core/rmnet_handlers.c
114:		nss_cb = rcu_dereference(rmnet_nss_callbacks);
466:		nss_cb = rcu_dereference(rmnet_nss_callbacks);

qsdk-5.4/qca/src/datarmnet/core/rmnet_descriptor.c
1539:		nss_cb = rcu_dereference(rmnet_nss_callbacks);

qsdk-5.4/qca/src/datarmnet/core/rmnet_config.c
185:	nss_cb = rcu_dereference(rmnet_nss_callbacks);
204:	nss_cb = rcu_dereference(rmnet_nss_callbacks);
837:struct rmnet_nss_cb *rmnet_nss_callbacks __rcu __read_mostly;
838:EXPORT_SYMBOL(rmnet_nss_callbacks);

qsdk-5.4/qca/src/data-kernel/drivers/rmnet-nss/rmnet_nss.c
425:	RCU_INIT_POINTER(rmnet_nss_callbacks, &rmnet_nss);
437:	RCU_INIT_POINTER(rmnet_nss_callbacks, NULL);

qsdk-5.4/qca/src/linux-5.4/include/linux/rmnet_nss.h
27:extern struct rmnet_nss_cb *rmnet_nss_callbacks;

6.6.34 seems to have fixed the issue. Thanks!

1 Like

Yup, it's the driver provided by Quectel themselves and according to them, it should be available on QSDK too. So does that mean these drivers I have are not compatible with 6.6.x kernel?

So I finally got some time to try it again while making sure those packages are present.

The behavior did change, slightly, but still far from desired one. The connection now did obeyed firewall+routing rules, but when comes to inter lan com I was not able to establish any connection between devices which were through the router and between tagged-untagged ports. While com solely between tagged-tagged and also untagged-untagged was working okay.

And when I say com, not even ping etc. The request for arp did arrive to the destination host, host does reply, but the reply is not bridged through the router back to the source host.

Pinging from the router itself is not in any way affected and works as expected.

Really dunno why it behaves so strangely.

i posted a working config some tiume back. search for it. a few post up there are working vlan configs as well.
nothing much we can do to help with the limited info presented.

if i had to guess. your fw/routing/gateway rules rules are out of wack.nothing to do with vlan config. cos your router can ping your host on your vlan already. so vlan tagging is working. maybe.

have fun

I mean, the thing is the same config works in the snapshot build. I obviously tried playing with the configs, including posted examples here :wink: The one using bridge in a bridge and so on...

I mean, the way I test it is just switching between snapshot build and nss build with following config:

config device
        option name 'br-lan'
        option type 'bridge'
        option ipv6 '0'
        list ports 'lan1'
        list ports 'lan2'
        list ports 'lan3'

config interface 'lan'
        option device 'br-lan.10'
        option proto 'static'
        option ip6assign '60'
        option ip6ifaceid 'eui64'
        option ip6hint '00'
        list ipaddr '192.168.144.1/24'

config interface 'lit'
        option device 'br-lan.20'
        option proto 'static'
        option ip6assign '64'
        option ip6ifaceid 'eui64'
        option ip6hint 'B4'
        list ipaddr '192.168.180.1/24'

config interface 'pub'
        option device 'br-lan.70'
        option proto 'static'
        option ip6assign '64'
        option ip6ifaceid 'eui64'
        option ip6hint 'D8'
        list ipaddr '192.168.216.1/24'

config interface 'wan'
        option device 'wan.100'
        option proto 'dhcp'

config interface 'wan6'
        option device 'wan.100'
        option proto 'dhcpv6'

config interface 'xdsl'
        option proto 'pppoe'
        option device 'wan.848'
        option username '*******'
        option password '*******'
        option ipv6 'auto'
        option ip4table 'default'
        option pppd_options 'ipv6cp-use-persistent'

config bridge-vlan
        option device 'br-lan'
        option vlan '10'
        list ports 'lan1:t*'
        list ports 'lan2:u*'
        list ports 'lan3:t*'

config bridge-vlan
        option device 'br-lan'
        option vlan '20'
        list ports 'lan1:t'
        list ports 'lan3:t'

config bridge-vlan
        option device 'br-lan'
        option vlan '70'
        list ports 'lan1:t'
        list ports 'lan3:t'

So the only variable which is changing is the build. :slight_smile:

try the format in the above post. the default openwrt of configuring vlans using bridge-vlan did not work for me on the nss builds.

its a back to basics way for defining vlans and from my limited experience works on the different makes of openwrt devices that i use.

a few days back there an updated threaded NAPI fix. maybe good to enable NAPI threading on qca-nss-drv again

3 Likes

I was debating about reverting my "revert" after seeing that. I haven't yet, mostly since my builds and uptime have been some of the most stable. Especially the UDP bug I had a while a back. Uptime has been over 2 weeks with zero kernel messages or connection drops.

I have a few updates to the nss packages and main repo I'll be pushing later today though.

3 Likes

i applied that qca-nss-drv patch (018) back and so far no issues saying this I never got the UDP issues.

2 Likes

UPDATE: 2024-06-23

Kernel: 6.6.35

OpenWRT NSS Fork:

date commit desc
2024-06-23 e38bbbed7b ath11k_nss: Fix patch conflict
2024-06-22 a1eff5b372 ath11k_nss: modularize nss-pbuf options
2024-06-22 6d56e60d7f ath11k_nss: Refresh patches
2024-06-20 dd5c59d06d ath11k_nss: drop upstreamed patch

NSS Packages:

date commit desc
2024-06-23 5c4eb1c qca-nss-drv: Improve nss_stats script
2024-06-23 b00980f treewide: Bump to consolidated release QSDK-12.5
2024-06-22 174edae nss-firmware: fix building for IPQ60XX/IPQ50XX

I took a stab at trying to get wwan offloaded, please test with branch NSS-12.5-6.x-wwan. Check commit notes for instructions.

4 Likes

Could you also make a separate branch for napi threads to test, please?

2 Likes

You're awesome, will look into this :slight_smile:

1 Like

Just pushed now.

@rmandrad , @JuliusBairaktaris. Would be curious to see how well this works with auto_scaling enabled for you. I know I said don't... but it's surprisingly been pretty stable last 5 days.

To set one time:

sysctl -w dev.nss.clock.auto_scale=1

Persist:

uci set pbuf.opt.auto_scale=1
uci commit pbuf
service qca-nss-pbuf restart

I've always seen this, even on current 6.6.35, but I never felt it caused any harm.

[58041.225004] softirq: huh, entered softirq 3 NET_RX 0000000077af5cdc with preempt_count 00000101, exited with 00000301?
[59652.316635] softirq: huh, entered softirq 3 NET_RX 0000000077af5cdc with preempt_count 00000101, exited with 00000301?
[59656.225369] softirq: huh, entered softirq 3 NET_RX 0000000077af5cdc with preempt_count 00000101, exited with 00000301?
[59656.229035] softirq: huh, entered softirq 3 NET_RX 0000000077af5cdc with preempt_count 00000101, exited with 00000301?
[59656.240726] softirq: huh, entered softirq 3 NET_RX 0000000077af5cdc with preempt_count 00000101, exited with 00000301?
[59656.245739] softirq: huh, entered softirq 3 NET_RX 0000000077af5cdc with preempt_count 00000101, exited with 00000301?
[59656.352127] softirq: huh, entered softirq 3 NET_RX 0000000077af5cdc with preempt_count 00000101, exited with 00000301?
[59656.366632] softirq: huh, entered softirq 3 NET_RX 0000000077af5cdc with preempt_count 00000101, exited with 00000301?
[59656.367717] softirq: huh, entered softirq 3 NET_RX 0000000077af5cdc with preempt_count 00000101, exited with 00000301?
[59656.376284] softirq: huh, entered softirq 3 NET_RX 0000000077af5cdc with preempt_count 00000100, exited with 00000300?
[59656.395954] softirq: huh, entered softirq 3 NET_RX 0000000077af5cdc with preempt_count 00000101, exited with 00000301?
[60106.737673] softirq: huh, entered softirq 3 NET_RX 0000000077af5cdc with preempt_count 00000101, exited with 00000301?
[60106.756127] softirq: huh, entered softirq 3 NET_RX 0000000077af5cdc with preempt_count 00000101, exited with 00000301?
[63117.589587] softirq: huh, entered softirq 3 NET_RX 0000000077af5cdc with preempt_count 00000101, exited with 00000301?
[63118.702566] softirq: huh, entered softirq 3 NET_RX 0000000077af5cdc with preempt_count 00000101, exited with 00000301?
[63344.695453] softirq: huh, entered softirq 3 NET_RX 0000000077af5cdc with preempt_count 00000101, exited with 00000301?
[63344.705418] softirq: huh, entered softirq 3 NET_RX 0000000077af5cdc with preempt_count 00000101, exited with 00000301?
[63391.164543] softirq: huh, entered softirq 3 NET_RX 0000000077af5cdc with preempt_count 00000101, exited with 00000301?
[63391.165848] softirq: huh, entered softirq 3 NET_RX 0000000077af5cdc with preempt_count 00000101, exited with 00000301?
[63391.235982] softirq: huh, entered softirq 3 NET_RX 0000000077af5cdc with preempt_count 00000101, exited with 00000301?

In order to use it we should just change feeds.conf.default to below, right?

NSS-12.5-K6.x-NAPI

Tested, the compilation was successful and the module is detected as it should however, for some reason Quectel-CM won't provide IP address for the interface.

Mon Jun 24 08:01:33 2024 kern.info kernel: [   13.682593] qmi_wwan_q 4-1:1.4: cdc-wdm0: USB WDM device
Mon Jun 24 08:01:33 2024 kern.info kernel: [   13.682854] qmi_wwan_q 4-1:1.4: Quectel RG500Q-EA work on RawIP mode
Mon Jun 24 08:01:33 2024 kern.info kernel: [   13.687261] qmi_wwan_q 4-1:1.4: rx_urb_size = 31744
Mon Jun 24 08:01:33 2024 kern.info kernel: [   13.694005] qmi_wwan_q 4-1:1.4 wwan0: register 'qmi_wwan_q' at usb-xhci-hcd.2.auto-1, RMNET/USB device, 3a:ff:46:f5:29:29
Mon Jun 24 08:01:33 2024 kern.info kernel: [   13.699468] net wwan0 wwan0_1: NSS context created
Mon Jun 24 08:01:33 2024 kern.info kernel: [   13.709222] net wwan0: qmap_register_device wwan0_1
Mon Jun 24 08:01:37 2024 daemon.notice netifd: 5g (4769): [06-24_08:01:37:106] QConnectManager_Linux_V1.6.5
Mon Jun 24 08:01:37 2024 daemon.notice netifd: 5g (4769): [06-24_08:01:37:106] Find /sys/bus/usb/devices/4-1 idVendor=0x2c7c idProduct=0x800, bus=0x004, dev=0x002
Mon Jun 24 08:01:37 2024 daemon.notice netifd: 5g (4769): [06-24_08:01:37:107] Auto find qmichannel = /dev/cdc-wdm0
Mon Jun 24 08:01:37 2024 daemon.notice netifd: 5g (4769): [06-24_08:01:37:107] Auto find usbnet_adapter = wwan0
Mon Jun 24 08:01:37 2024 daemon.notice netifd: 5g (4769): [06-24_08:01:37:107] netcard driver = qmi_wwan_q, driver version = V1.2.2
Mon Jun 24 08:01:37 2024 daemon.notice netifd: 5g (4769): [06-24_08:01:37:108] qmap_mode = 1, qmap_version = 9, qmap_size = 31744, muxid = 0x81, qmap_netcard = wwan0_1
Mon Jun 24 08:01:37 2024 daemon.notice netifd: 5g (4769): [06-24_08:01:37:108] Modem works in QMI mode
Mon Jun 24 08:01:37 2024 daemon.notice netifd: 5g (4769): [06-24_08:01:37:116] cdc_wdm_fd = 7
Mon Jun 24 08:01:39 2024 daemon.notice netifd: 5g (4769): [06-24_08:01:39:721] requestGetSIMStatus SIMStatus: SIM_READY
Mon Jun 24 08:01:39 2024 daemon.notice netifd: 5g (4769): [06-24_08:01:39:785] requestGetProfile[pdp:1 index:1] internet///0/IPV4
Mon Jun 24 08:01:39 2024 daemon.notice netifd: 5g (4769): [06-24_08:01:39:785] requestSetProfile[pdp:1 index:1] internet///0/IPV4
Mon Jun 24 08:01:39 2024 daemon.notice netifd: 5g (4769): [06-24_08:01:39:786] no need to set skip the rest
Mon Jun 24 08:01:39 2024 daemon.notice netifd: 5g (4769): [06-24_08:01:39:818] requestRegistrationState2 MCC: 515, MNC: 3, PS: Attached, DataCap: 5G_NSA
Mon Jun 24 08:01:39 2024 daemon.notice netifd: 5g (4769): [06-24_08:01:39:851] requestQueryDataCall IPv4ConnectionStatus: DISCONNECTED
Mon Jun 24 08:01:39 2024 daemon.notice netifd: 5g (4769): [06-24_08:01:39:851] ip link set dev wwan0 down
Mon Jun 24 08:01:39 2024 daemon.notice netifd: 5g (4769): [06-24_08:01:39:854] ip addr flush dev wwan0_1
Mon Jun 24 08:01:39 2024 daemon.notice netifd: 5g (4769): [06-24_08:01:39:856] ip link set dev wwan0_1 down
Mon Jun 24 08:01:39 2024 daemon.notice netifd: 5g (4769): [06-24_08:01:39:881] requestSetupDataCall WdsConnectionIPv4Handle: 0x777c7510
Mon Jun 24 08:01:40 2024 daemon.notice netifd: 5g (4769): [06-24_08:01:40:010] ip link set dev wwan0 up
Mon Jun 24 08:01:40 2024 kern.info kernel: [   58.899399] net wwan0: link_state 0x0 -> 0x1
Mon Jun 24 08:01:40 2024 daemon.notice netifd: 5g (4769): RTNETLINK answers: Address not available
Mon Jun 24 08:01:40 2024 daemon.notice netifd: 5g (4769): [06-24_08:01:40:012] ip link set dev wwan0_1 up
Mon Jun 24 08:01:40 2024 daemon.notice netifd: 5g (4769): RTNETLINK answers: Network is down
Mon Jun 24 08:01:40 2024 daemon.notice netifd: 5g (4769): [06-24_08:01:40:020] busybox udhcpc -f -n -q -t 5 -i wwan0_1
Mon Jun 24 08:01:40 2024 daemon.notice netifd: 5g (4769): udhcpc: started, v1.36.1
Mon Jun 24 08:02:10 2024 daemon.notice netifd: 5g (4769): udhcpc: read error: Network is down, reopening socket
Mon Jun 24 08:02:13 2024 daemon.notice netifd: 5g (4769): udhcpc: read error: Network is down, reopening socket
Mon Jun 24 08:02:16 2024 daemon.notice netifd: 5g (4769): udhcpc: read error: Network is down, reopening socket
Mon Jun 24 08:02:19 2024 daemon.notice netifd: 5g (4769): udhcpc: read error: Network is down, reopening socket

I had also tried just using "qmi" as protocol and that also seems to experience the same issue with the IP.

I had better results using the packages and drivers from this repo if you can look into it. https://github.com/immortalwrt/wwan-packages

Correct, so something like:

src-git nss_packages https://github.com/qosmio/nss-packages.git;NSS-12.5-K6.x-NAPI

Hmm, not sure, tbh it's my first time working with wwan type drivers. Looked at the module list from the OEM fw dump, and tried to work backwards.

So from immortalwrt repo you're just building:

quectel-cm
kmod-usb-net-qmi-wwan-quectel (qmi_wwan_q.ko)
kmod-mhi-pcie (pcie_mhi.ko)

Correct?

It's super confusing because at least 4 different repos I searched in, organize the modules very differently. The ones I tried to piece meal from were coolsnowwolf/lede and immortalwrt/wwan-packages.

I see rmnet_nss.c inside two packages quectel_QMI_WWAN and quectel_MHI. But no references for the files exists in any of the Makefiles. quectel_MHI also shows code that references a lot of nss related code, but the makefile skips over it completely to build just devices/mhi_netdev_quectel:

ifeq (1,1)
 pcie_mhi-objs += devices/mhi_netdev_quectel.o
else
 pcie_mhi-objs += devices/mhi_netdev.o
 pcie_mhi-objs += devices/rmnet_handler.o
endif

Not sure if devices/mhi_netdev_quectel is all that's needed for offloading. The other directory shows that it's related to "RmNet MAP driver multiplexing and aggregation protocol (MAP)", which I'm assuming is for fully offloading. ecm has hooks for RAWIP which I see in a few places under rmnet directory.

I recall from your post you had an issue with undefined rmnet_nss_callbacks symbols, you could try just applying the patch directly to rmnet_nss.c with immortalwrt's repo.

--- a/rmnet_nss.c
+++ b/rmnet_nss.c
@@ -31,8 +31,8 @@ struct rmnet_nss_cb {
         int (*nss_tx)(struct sk_buff *skb);
 };

-extern struct rmnet_nss_cb *rmnet_nss_callbacks;
-
+struct rmnet_nss_cb *rmnet_nss_callbacks;
+EXPORT_SYMBOL(rmnet_nss_callbacks);
 #endif

 #define RMNET_NSS_HASH_BITS 8

Anyhow... I'm not too sure how much I can help with this since it's pretty specialized, and not i'm not too well versed in wwan protocols.