IPQ806x NSS Drivers

I'm getting these warnings:

WARNING: Makefile 'package/qca/qca-nss-drv/Makefile' has a dependency on 'kmod-qca-nss-dp', which does not exist
WARNING: Makefile 'package/qca/qca-nss-ecm/Makefile' has a dependency on 'kmod-qca-ovsmgr', which does not exist
WARNING: Makefile 'package/qca/qca-nss-ecm/Makefile' has a dependency on 'kmod-qca-ovsmgr', which does not exist
WARNING: Makefile 'package/qca/qca-nss-ecm/Makefile' has a dependency on 'kmod-qca-ovsmgr', which does not exist
WARNING: Makefile 'package/qca/qca-nss-ecm/Makefile' has a dependency on 'kmod-qca-hyfi-bridge', which does not exist
WARNING: Makefile 'package/qca/qca-nss-ecm/Makefile' has a dependency on 'kmod-qca-mcs', which does not exist
WARNING: Makefile 'package/qca/qca-nss-ecm/Makefile' has a dependency on 'kmod-qca-ovsmgr', which does not exist
WARNING: Makefile 'package/qca/qca-nss-ecm/Makefile' has a dependency on 'kmod-qca-hyfi-bridge', which does not exist
WARNING: Makefile 'package/qca/qca-nss-ecm/Makefile' has a dependency on 'kmod-qca-mcs', which does not exist

Is this critical?

Targeting to the Linksys EA7500 V1.

Not critical - those warnings are present every time I build. Let us know if it doesn’t build or doesn’t boot.

Well, it builds:

But there are some bin files for the R7800, still.

How can I get sure the EA7500 V1 binaries are with NSS drivers turned on?

Bad news: with a sysupgrade binary, upgrading my current version, it does not boot.

After 3 failed reboots, reverted to the previous flashed versions.

Any ability to capture some logs from your ea7500 v1 for @Ansuel to see if there is a modification to get it to work?

Additionally can you post your config.buildinfo from your bin folder to see if there are any config issues?

I can't get any logs, since the router doesn't boot correctly, I guess.

The config.buildinfo:

CONFIG_TARGET_ipq806x=y
CONFIG_TARGET_ipq806x_generic=y
CONFIG_TARGET_ipq806x_generic_DEVICE_linksys_ea7500-v1=y
CONFIG_BUSYBOX_CUSTOM=y
CONFIG_BUILD_PATENTED=y
CONFIG_BUSYBOX_CONFIG_FEATURE_EDITING_SAVEHISTORY=y
CONFIG_BUSYBOX_CONFIG_FEATURE_EDITING_SAVE_ON_EXIT=y
CONFIG_BUSYBOX_CONFIG_FEATURE_LESS_FLAGS=y
CONFIG_BUSYBOX_CONFIG_FEATURE_LESS_REGEXP=y
CONFIG_BUSYBOX_CONFIG_FEATURE_LESS_WINCH=y
CONFIG_DROPBEAR_ECC=y
CONFIG_IMAGEOPT=y
CONFIG_OPENSSL_ENGINE=y
CONFIG_OPENSSL_PREFER_CHACHA_OVER_GCM=y
CONFIG_OPENSSL_WITH_ASM=y
CONFIG_OPENSSL_WITH_CHACHA_POLY1305=y
CONFIG_OPENSSL_WITH_CMS=y
CONFIG_OPENSSL_WITH_DEPRECATED=y
CONFIG_OPENSSL_WITH_ERROR_MESSAGES=y
CONFIG_OPENSSL_WITH_PSK=y
CONFIG_OPENSSL_WITH_SRP=y
CONFIG_OPENSSL_WITH_TLS13=y
CONFIG_PACKAGE_6in4=y
CONFIG_PACKAGE_6rd=y
CONFIG_PACKAGE_6to4=y
CONFIG_PACKAGE_MAC80211_NSS_SUPPORT=y
CONFIG_PACKAGE_NTFS-3G_HAS_PROBE=y
CONFIG_PACKAGE_ath10k-firmware-qca9984-ct=y
# CONFIG_PACKAGE_ath10k-firmware-qca99x0-ct is not set
CONFIG_PACKAGE_block-mount=y
CONFIG_PACKAGE_ca-certificates=y
CONFIG_PACKAGE_cgi-io=y
CONFIG_PACKAGE_collectd=y
CONFIG_PACKAGE_collectd-mod-conntrack=y
CONFIG_PACKAGE_collectd-mod-cpu=y
CONFIG_PACKAGE_collectd-mod-cpufreq=y
CONFIG_PACKAGE_collectd-mod-entropy=y
CONFIG_PACKAGE_collectd-mod-exec=y
CONFIG_PACKAGE_collectd-mod-interface=y
CONFIG_PACKAGE_collectd-mod-iwinfo=y
CONFIG_PACKAGE_collectd-mod-load=y
CONFIG_PACKAGE_collectd-mod-memory=y
CONFIG_PACKAGE_collectd-mod-network=y
CONFIG_PACKAGE_collectd-mod-ping=y
CONFIG_PACKAGE_collectd-mod-rrdtool=y
CONFIG_PACKAGE_collectd-mod-sqm=y
CONFIG_PACKAGE_collectd-mod-thermal=y
CONFIG_PACKAGE_collectd-mod-uptime=y
CONFIG_PACKAGE_dawn=y
CONFIG_PACKAGE_hostapd-utils=y
CONFIG_PACKAGE_ip6tables-mod-nat=y
CONFIG_PACKAGE_iptables-mod-conntrack-extra=y
CONFIG_PACKAGE_iptables-mod-extra=y
CONFIG_PACKAGE_iptables-mod-ipopt=y
CONFIG_PACKAGE_iptables-mod-physdev=y
CONFIG_PACKAGE_irqbalance=y
CONFIG_PACKAGE_kmod-br-netfilter=y
CONFIG_PACKAGE_kmod-crypto-acompress=y
CONFIG_PACKAGE_kmod-crypto-aead=y
CONFIG_PACKAGE_kmod-crypto-arc4=y
CONFIG_PACKAGE_kmod-crypto-authenc=y
CONFIG_PACKAGE_kmod-crypto-cbc=y
CONFIG_PACKAGE_kmod-crypto-crc32c=y
CONFIG_PACKAGE_kmod-crypto-deflate=y
CONFIG_PACKAGE_kmod-crypto-des=y
CONFIG_PACKAGE_kmod-crypto-ecb=y
CONFIG_PACKAGE_kmod-crypto-echainiv=y
CONFIG_PACKAGE_kmod-crypto-hash=y
CONFIG_PACKAGE_kmod-crypto-hmac=y
CONFIG_PACKAGE_kmod-crypto-manager=y
CONFIG_PACKAGE_kmod-crypto-md4=y
CONFIG_PACKAGE_kmod-crypto-md5=y
CONFIG_PACKAGE_kmod-crypto-null=y
CONFIG_PACKAGE_kmod-crypto-pcompress=y
CONFIG_PACKAGE_kmod-crypto-sha1=y
CONFIG_PACKAGE_kmod-crypto-sha256=y
CONFIG_PACKAGE_kmod-fs-cifs=y
CONFIG_PACKAGE_kmod-fs-exfat=y
CONFIG_PACKAGE_kmod-fs-ext4=y
CONFIG_PACKAGE_kmod-fs-hfsplus=y
CONFIG_PACKAGE_kmod-fs-msdos=y
CONFIG_PACKAGE_kmod-fs-vfat=y
CONFIG_PACKAGE_kmod-fuse=y
CONFIG_PACKAGE_kmod-ifb=y
CONFIG_PACKAGE_kmod-ipsec=y
CONFIG_PACKAGE_kmod-ipt-conntrack-extra=y
CONFIG_PACKAGE_kmod-ipt-extra=y
CONFIG_PACKAGE_kmod-ipt-ipopt=y
CONFIG_PACKAGE_kmod-ipt-nat6=y
CONFIG_PACKAGE_kmod-ipt-physdev=y
CONFIG_PACKAGE_kmod-ipt-raw=y
CONFIG_PACKAGE_kmod-iptunnel=y
CONFIG_PACKAGE_kmod-iptunnel4=y
CONFIG_PACKAGE_kmod-ledtrig-default-on=y
CONFIG_PACKAGE_kmod-ledtrig-heartbeat=y
CONFIG_PACKAGE_kmod-ledtrig-netdev=y
CONFIG_PACKAGE_kmod-ledtrig-timer=y
CONFIG_PACKAGE_kmod-lib-crc16=y
CONFIG_PACKAGE_kmod-lib-zlib-deflate=y
CONFIG_PACKAGE_kmod-lib-zlib-inflate=y
CONFIG_PACKAGE_kmod-nf-conntrack-netlink=y
CONFIG_PACKAGE_kmod-nf-nat6=y
CONFIG_PACKAGE_kmod-nfnetlink=y
CONFIG_PACKAGE_kmod-nls-cp1250=y
CONFIG_PACKAGE_kmod-nls-cp437=y
CONFIG_PACKAGE_kmod-nls-cp850=y
CONFIG_PACKAGE_kmod-nls-iso8859-1=y
CONFIG_PACKAGE_kmod-nls-iso8859-15=y
CONFIG_PACKAGE_kmod-nls-utf8=y
CONFIG_PACKAGE_kmod-nss-ifb=y
CONFIG_PACKAGE_kmod-qca-nss-drv=y
CONFIG_PACKAGE_kmod-qca-nss-drv-qdisc=y
CONFIG_PACKAGE_kmod-qca-nss-ecm-standard=y
CONFIG_PACKAGE_kmod-qca-nss-gmac=y
CONFIG_PACKAGE_kmod-sched-cake=y
CONFIG_PACKAGE_kmod-sched-core=y
CONFIG_PACKAGE_kmod-sit=y
CONFIG_PACKAGE_kmod-usb-storage=y
CONFIG_PACKAGE_libblkid=y
CONFIG_PACKAGE_libelf=y
CONFIG_PACKAGE_libgcrypt=y
CONFIG_PACKAGE_libgpg-error=y
CONFIG_PACKAGE_libiwinfo-lua=y
CONFIG_PACKAGE_libltdl=y
CONFIG_PACKAGE_liblua=y
CONFIG_PACKAGE_liblucihttp=y
CONFIG_PACKAGE_liblucihttp-lua=y
CONFIG_PACKAGE_libopenssl=y
CONFIG_PACKAGE_libopenssl-conf=y
CONFIG_PACKAGE_liboping=y
CONFIG_PACKAGE_librrd1=y
CONFIG_PACKAGE_librt=y
CONFIG_PACKAGE_libubus-lua=y
CONFIG_PACKAGE_libustream-openssl=y
# CONFIG_PACKAGE_libustream-wolfssl is not set
CONFIG_PACKAGE_libuuid=y
# CONFIG_PACKAGE_libwolfssl is not set
CONFIG_PACKAGE_lua=y
CONFIG_PACKAGE_luci=y
CONFIG_PACKAGE_luci-app-commands=y
CONFIG_PACKAGE_luci-app-dawn=y
CONFIG_PACKAGE_luci-app-firewall=y
CONFIG_PACKAGE_luci-app-nlbwmon=y
CONFIG_PACKAGE_luci-app-opkg=y
CONFIG_PACKAGE_luci-app-sqm=y
CONFIG_PACKAGE_luci-app-statistics=y
CONFIG_PACKAGE_luci-base=y
CONFIG_PACKAGE_luci-compat=y
CONFIG_PACKAGE_luci-lib-base=y
CONFIG_PACKAGE_luci-lib-ip=y
CONFIG_PACKAGE_luci-lib-json=y
CONFIG_PACKAGE_luci-lib-jsonc=y
CONFIG_PACKAGE_luci-lib-nixio=y
CONFIG_PACKAGE_luci-mod-admin-full=y
CONFIG_PACKAGE_luci-mod-network=y
CONFIG_PACKAGE_luci-mod-status=y
CONFIG_PACKAGE_luci-mod-system=y
CONFIG_PACKAGE_luci-proto-ipv6=y
CONFIG_PACKAGE_luci-proto-ppp=y
CONFIG_PACKAGE_luci-ssl-openssl=y
CONFIG_PACKAGE_luci-theme-bootstrap=y
CONFIG_PACKAGE_nlbwmon=y
CONFIG_PACKAGE_ntfs-3g=y
CONFIG_PACKAGE_openssl-util=y
CONFIG_PACKAGE_rpcd=y
CONFIG_PACKAGE_rpcd-mod-file=y
CONFIG_PACKAGE_rpcd-mod-iwinfo=y
CONFIG_PACKAGE_rpcd-mod-luci=y
CONFIG_PACKAGE_rpcd-mod-rrdns=y
CONFIG_PACKAGE_rrdtool1=y
CONFIG_PACKAGE_sqm-scripts=y
CONFIG_PACKAGE_tc=y
CONFIG_PACKAGE_uhttpd=y
CONFIG_PACKAGE_uhttpd-mod-ubus=y
CONFIG_PACKAGE_umdns=y
# CONFIG_PACKAGE_wpad-basic-wolfssl is not set
CONFIG_PACKAGE_wpad-openssl=y
CONFIG_PACKAGE_zlib=y
CONFIG_PREINITOPT=y
CONFIG_TARGET_PREINIT_TIMEOUT=5
CONFIG_WPA_MSG_MIN_PRIORITY=4
# CONFIG_WPA_WOLFSSL is not set

PS: This router is only supported on the snapshots.

PS2: I don't know if this is relevant or not, but with the November 12th snapshops, I can't get the WAN to work. Not even with static IP address. Had to return to the November 9th one:

https://pastebin.com/yN7F97Yz

I guess it'll probably be much easier waiting for the OpenVPN kernel module and hack that to use the NSS crypto engine.

Just let you know the OpenVPN kernel module is out and under in heavy development. There are below limiations as below according to README.

== Limitations ==
This is a list of current limitations which are planned to be removed as we move forward:

  • Only client mode supported
  • Only AES-256-GCM and 'none' (with no auth) supported

https://www.mail-archive.com/openvpn-devel@lists.sourceforge.net/msg21065.html

Thanks for the update @Tony.He.

Good to know that this finally gets some traction. The ipq806x (and I believe for ipx807x) NSS firmware currently doesn't support GCM mode tho. I guess the developers concentrated on GCM due to it being the recommended mode currently. Hopefully adding AES-CBC-SHA-HMAC mode should be straightforward.

At the moment I don't really have much time to hack router firmware due to work. Hopefully I can restart this sometime in future soon.

1 Like

Do you think the openvpn kmod will give better perf as it will skip the userspace problem?
Also I wonder if wireguard performance can benefit of the crypto core.

Hi @Tyco89,

To follow up and to raise awareness for others on the spontaneous restarts, there seem to be an underlying bug not related to NSS that may cause router restarts.

I had 17d uptime with my NSS build before a spontaneous restart yesterday.
The crash logs points directly to the already created bug report below.

The workaround mentioned in that bug report is to use performance governor.

Your restarts may be from a different cause but it might be worth a try to change to performance governor if you are still facing frequent restart problem and not already using the performance governor.

As for the fix for that reported bug, it may take quite some time to get fixed...

Definitely. From what I’ve tested with the NSS crypto core, moving from user space to kernel space for the data plane results in 4X improvement in thruput. With further optimization, it should go higher (I hope) with better integration with the NSS firmware.

I briefly looked at the ovpn-dco code. It looks like they are creating a separate network interface and not using the Linux tun interface, which makes sense since the tun interface is mainly for user space processes to send network packets. I think using the new approach should make integration with the NSS a lot easier using the NSS virtual interface APIs.

Unfortunately the NSS firmware doesn’t support the crypto algorithm or the authentication mode that WireGuard is using. Good news is that I could get about 200mbps (if my memory serves) thruput between a mt7621 and a bcm4708 SoC router over a WAN link between two ISPs across two countries. Using ipq806x SoC routers for both ends would definitely results in higher WireGuard thruput.

1 Like

Thank you @Gram! I'll try and report back. I had the governor set to on-demand

Speaking of NSS and VPN, any idea if ipsec is or can be offloaded to NSS?
I'm currently running a separate VM for VPN related stuff (to keeps the route's CPU utilization low)
Was wondering if R7800 could handle the VPN part as well with the help of NSS

I run a variant of hnyman's build on my R7800 (i won't tag him as i don't think it's anything to do with his build), with some minor changes of my own, among them support for jumbo frames and using DSA instead of swconfig.

I tried merging this in but could not get it to work successfully...I left the DSA config in place initially (including the DTS changes) which resulted in none of the ethernet ports starting, luckily wireless still worked. I reverted the DSA changes in the DTS and in /etc/config, still no joy. Then I did a factory reset and the LAN ports came up, but still no WAN port. In luci it complains that the device is not present, in the kernel log you can see eth1 start up, but not eth0.

I'm afraid all this is a bit beyond me, anything obvious I've missed?

this is my tree: https://github.com/facboy/openwrt/tree/hnyman-ct-nss
and the kernel log: https://gist.github.com/facboy/854d09d45ff55b1f4ced23c8893449ef

nss doesn't like the dsa system... For ethernet you are stuck with swconfig for now.

Wish I knew why NSS enabled image won't boot for me. The config i use is basically the same config as hnyman use in his builds now when he enabled wireguard support.

I'm stuck also, no boot at my EA7500 V1.

My build is pretty similar to hynman’s. I have my diffconfig posted if you want to compare - or if it is helpful you could try my sysupgrade and add on your packages (just to try it out).

There must be a package or dependency that it doesn’t like.

Still getting spontaneous reboots. Anyone else? It reboots quickly and comes back, but still reboots periodically.

yes, i switched back to swconfig, but eth0 still doesn't come up.