IPQ806x NSS Drivers

Thanks @quarky
Can anyone advice (i.e. using priv message) where can I get a modern/appropriate firmware version?

Any possibility to roll up the version 6 or 10 in to an openwrt software package? Something that would capture the best of all the repositories, jointly work on, and be easy to apply / remove?

Hello!
I want to thank all guys who actively worked (and probably still working) on making this dream a reality!
Great job :slight_smile:
I loaded into my router an image with Ansuel's work (QSDK v.10.0) & Sqter's patches - it works beautifully for almost 5 hours, and I can saturate my 500 Mbit broadband using a Wifi, with relatively small CPU utilisation
I might be wrong, but after I enabled VHT160 and used a Wifi client with 2x2 card, under heavy tests, some strange exceptions started to show up in the device's log:
https://pastebin.com/AcnRjmdQ
Could anyone more experienced advise what's causing it? Is it a VHT160 + Client's 2x2 wifi? Or is it my VAP? Or something else?

That is not possible, legal redistribution questions aside[0], switching between vanilla kernel support and NSS offloading is highly invasive, requiring DTS changes and more (so you do need different kernel-/ firmware builds with vastly different configurations and patches applied). It's not a case of install package a, b and c, tick checkbox x in the webinterface, far from it.

--
[0] which are not answered easily.

Actually if you can build an image yourself the installation is pretty easy and quick. But yes an installable package is not possible. We can still produce firmware with empty firmware and just make the user upload it or save on the mtd. (If I'm not wrong the bin load position can be set in the uci config file)

@Ansuel @quarky @sqter

I want try nss driver on my ea7500 v1
ipq806x: add support for Linksys EA7500 V1

but I don't know how to modify "qcom-ipq8064-ea7500-v1.dts" or other related files
Could you help me
thanks

// SPDX-License-Identifier: GPL-2.0-or-later OR MIT
#include "qcom-ipq8064-v2.0.dtsi"

#include <dt-bindings/input/input.h>

/ {
	model = "Linksys EA7500 V1 WiFi Router";
	compatible = "linksys,ea7500-v1", "qcom,ipq8064";

	memory@0 {
		reg = <0x42000000 0xe000000>;
		device_type = "memory";
	};

	reserved-memory {
		#address-cells = <1>;
		#size-cells = <1>;
		ranges;
		rsvd@41200000 {
			reg = <0x41200000 0x300000>;
			no-map;
		};
	};

	aliases {
		serial0 = &gsbi4_serial;

		led-boot = &led_power;
		led-failsafe = &led_power;
		led-running = &led_power;
		led-upgrade = &led_power;
	};

	chosen {
		bootargs = "console=ttyMSM0,115200n8";
		stdout-path = "serial0:115200n8";
		append-rootblock = "ubi.mtd=";  /* append to bootargs adding the root deviceblock nbr from bootloader */
		find-rootblock = "ubi.mtd=";  /* look for root deviceblock nbr in this bootarg */
	};

	keys {
		compatible = "gpio-keys";
		pinctrl-0 = <&button_pins>;
		pinctrl-names = "default";

		reset {
			label = "reset";
			gpios = <&qcom_pinmux 68 GPIO_ACTIVE_LOW>;
			linux,code = <KEY_RESTART>;
		};

		wps {
			label = "wps";
			gpios = <&qcom_pinmux 65 GPIO_ACTIVE_LOW>;
			linux,code = <KEY_WPS_BUTTON>;
		};
	};

	leds {
		compatible = "gpio-leds";
		pinctrl-0 = <&led_pins>;
		pinctrl-names = "default";

		led_power: power {
			label = "ea7500-v1:white:power";
			gpios = <&qcom_pinmux 6 GPIO_ACTIVE_LOW>;
			default-state = "keep";
		};
	};
};

&qcom_pinmux {
	button_pins: button_pins {
		mux {
			pins = "gpio65", "gpio68";
			function = "gpio";
			drive-strength = <2>;
			bias-pull-up;
		};
	};

	led_pins: led_pins {
		mux {
			pins = "gpio6";
			function = "gpio";
			drive-strength = <2>;
			bias-pull-up;
		};
	};
};

&gsbi4 {
	qcom,mode = <GSBI_PROT_I2C_UART>;
	status = "okay";
	serial@16340000 {
		status = "okay";
	};
	/*
	* The i2c device on gsbi4 should not be enabled.
	* On ipq806x designs gsbi4 i2c is meant for exclusive
	* RPM usage. Turning this on in kernel manifests as
	* i2c failure for the RPM.
	*/
};

&usb3_0 {
	status = "okay";
};

&usb3_1 {
	status = "okay";
};

&pcie0 {
	status = "okay";
	force_gen1 = <1>;
};

&pcie1 {
	status = "okay";
};

&pcie2 {
	status = "okay";
};

&nand_controller {
	status = "okay";

	pinctrl-0 = <&nand_pins>;
	pinctrl-names = "default";

	nand@0 {
		reg = <0>;
		compatible = "qcom,nandcs";

		nand-ecc-strength = <4>;
		nand-bus-width = <8>;
		nand-ecc-step-size = <512>;

		partitions {
			compatible = "fixed-partitions";
			#address-cells = <1>;
			#size-cells = <1>;

			partition@0 {
				label = "SBL1";
				reg = <0x0000000 0x0040000>;
				read-only;
			};

			partition@40000 {
				label = "MIBIB";
				reg = <0x0040000 0x0140000>;
				read-only;
			};

			partition@180000 {
				label = "SBL2";
				reg = <0x0180000 0x0140000>;
				read-only;
			};

			partition@2c0000 {
				label = "SBL3";
				reg = <0x02c0000 0x0280000>;
				read-only;
			};

			partition@540000 {
				label = "DDRCONFIG";
				reg = <0x0540000 0x0120000>;
				read-only;
			};

			partition@660000 {
				label = "SSD";
				reg = <0x0660000 0x0120000>;
				read-only;
			};

			partition@780000 {
				label = "TZ";
				reg = <0x0780000 0x0280000>;
				read-only;
			};

			partition@a00000 {
				label = "RPM";
				reg = <0x0a00000 0x0280000>;
				read-only;
			};

			art: partition@c80000 {
				label = "art";
				reg = <0x0c80000 0x0140000>;
				read-only;
			};

			partition@dc0000 {
				label = "APPSBL";
				reg = <0x0dc0000 0x0100000>;
				read-only;
			};

			partition@ec0000 {
				label = "u_env";
				reg = <0x0ec0000 0x0040000>;
			};

			partition@f00000 {
				label = "s_env";
				reg = <0x0f00000 0x0040000>;
			};

			partition@f40000 {
				label = "devinfo";
				reg = <0x0f40000 0x0040000>;
			};

			partition@f80000 {
				label = "kernel1";
				reg = <0x0f80000 0x2800000>;  /* 3 MB spill to rootfs*/
			};

			partition@1280000 {
				label = "rootfs1";
				reg = <0x1280000 0x2500000>;
			};

			partition@3780000 {
				label = "kernel2";
				reg = <0x3780000 0x2800000>;
			};

			partition@3a80000 {
				label = "rootfs2";
				reg = <0x3a80000 0x2500000>;
			};

			partition@5f80000 {
				label = "sysdiag";
				reg = <0x5f80000 0x100000>;
			};

			partition@6080000 {
				label = "syscfg";
				reg = <0x6080000 0x1f80000>;
			};
		};
	};
};

&mdio0 {
	status = "okay";

	pinctrl-0 = <&mdio0_pins>;
	pinctrl-names = "default";

	phy0: ethernet-phy@0 {
		reg = <0>;
		qca,ar8327-initvals = <
			0x00004 0x7600000   /* PAD0_MODE */
			0x00008 0x1000000   /* PAD5_MODE */
			0x0000c 0x80        /* PAD6_MODE */
			0x00010 0x2613a0    /* PWS_REG */
			0x000e4 0x6a545     /* MAC_POWER_SEL */
			0x000e0 0xc74164de  /* SGMII_CTRL */
			0x0007c 0x4e        /* PORT0_STATUS */
			0x00094 0x4e        /* PORT6_STATUS */
			>;
	};
};

&gmac1 {
	status = "okay";
	phy-mode = "rgmii";
	qcom,id = <1>;

	pinctrl-0 = <&rgmii2_pins>;
	pinctrl-names = "default";

	fixed-link {
		speed = <1000>;
		full-duplex;
	};
};

&gmac2 {
	status = "okay";
	phy-mode = "sgmii";
	qcom,id = <2>;

	fixed-link {
		speed = <1000>;
		full-duplex;
	};
};

&adm_dma {
	status = "okay";
};

Have you successfully loaded OpenWrt to your EA7500v1? If not, that should be your first step. You should only try NSS drivers only after you confirmed that OpenWrt runs reliably in you EA7500v1.

Once you make the build reliable, all you need to do is to change he gmac1 & gmac2 DTS nodes of your EA7500v1 to use the QCA GMAC drivers and the rest will startup automatically, if you select all related NSS drivers.

I do successfully loaded OpenWrt
but I got error while buiding ea7500v1 on @quarky and @sqter ipq806x-nss-qsdk10
I will show the error later
thanks

here are the error log

  SHIPPED arch/arm/boot/compressed/fdt_wip.c
  CC      arch/arm/boot/compressed/fdt_wip.o
  SHIPPED arch/arm/boot/compressed/fdt.c
  CC      arch/arm/boot/compressed/fdt.o
  CC      arch/arm/boot/compressed/atags_to_fdt.o
arch/arm/boot/compressed/atags_to_fdt.c: In function 'append_rootblock':
arch/arm/boot/compressed/atags_to_fdt.c:104:17: warning: assignment discards 'const' qualifier from pointer target type [-Wdiscarded-qualifiers]
  find_rootblock = getprop(fdt, "/chosen", "find-rootblock", &l);
                 ^
arch/arm/boot/compressed/atags_to_fdt.c: In function 'atags_to_fdt':
arch/arm/boot/compressed/atags_to_fdt.c:219:11: warning: unused variable 'memsize' [-Wunused-variable]
  int ret, memsize;
           ^~~~~~~
arch/arm/boot/compressed/atags_to_fdt.c:218:6: warning: unused variable 'memcount' [-Wunused-variable]
  int memcount = 0;
      ^~~~~~~~
arch/arm/boot/compressed/atags_to_fdt.c:217:11: warning: unused variable 'mem_reg_property' [-Wunused-variable]
  uint32_t mem_reg_property[2 * 2 * NR_BANKS];
           ^~~~~~~~~~~~~~~~
At top level:
arch/arm/boot/compressed/atags_to_fdt.c:186:13: warning: 'hex_str' defined but not used [-Wunused-function]
 static void hex_str(char *out, uint32_t value)
             ^~~~~~~
arch/arm/boot/compressed/atags_to_fdt.c:61:17: warning: 'get_cell_size' defined but not used [-Wunused-function]
 static uint32_t get_cell_size(const void *fdt)
                 ^~~~~~~~~~~~~
arch/arm/boot/compressed/atags_to_fdt.c:41:12: warning: 'setprop_cell' defined but not used [-Wunused-function]
 static int setprop_cell(void *fdt, const char *node_path,
            ^~~~~~~~~~~~
arch/arm/boot/compressed/atags_to_fdt.c:23:12: warning: 'setprop' defined but not used [-Wunused-function]
 static int setprop(void *fdt, const char *node_path, const char *property,
            ^~~~~~~
arch/arm/boot/compressed/atags_to_fdt.c: In function 'merge_fdt_bootargs':
arch/arm/boot/compressed/atags_to_fdt.c:184:1: warning: the frame size of 1040 bytes is larger than 1024 bytes [-Wframe-larger-than=]
 }
 ^
  SHIPPED arch/arm/boot/compressed/lib1funcs.S
  AS      arch/arm/boot/compressed/lib1funcs.o
  SHIPPED arch/arm/boot/compressed/ashldi3.S
  AS      arch/arm/boot/compressed/ashldi3.o
  SHIPPED arch/arm/boot/compressed/bswapsdi2.S
  AS      arch/arm/boot/compressed/bswapsdi2.o
  LD      arch/arm/boot/compressed/vmlinux
  OBJCOPY arch/arm/boot/zImage
  Kernel: arch/arm/boot/zImage is ready
make[5]: Nothing to be done for 'Image'.
  DTC     arch/arm/boot/dts/qcom-ipq8064-ea7500-v1.dtb
arch/arm/boot/dts/qcom-ipq8064.dtsi:1321.45-1338.5: ERROR (phandle_references): /soc/nand-controller@1ac00000: Reference to non-existent node or label "nand_pins"

  also defined at arch/arm/boot/dts/qcom-ipq8064-ea7500-v1.dts:127.18-252.3
arch/arm/boot/dts/qcom-ipq8064.dtsi:1356.24-1366.5: ERROR (phandle_references): /soc/mdio@37000000: Reference to non-existent node or label "mdio0_pins"

  also defined at arch/arm/boot/dts/qcom-ipq8064-ea7500-v1.dts:254.8-273.3
arch/arm/boot/dts/qcom-ipq8064.dtsi:1397.28-1418.5: ERROR (phandle_references): /soc/ethernet@37200000: Reference to non-existent node or label "rgmii2_pins"

  also defined at arch/arm/boot/dts/qcom-ipq8064-ea7500-v1.dts:275.8-287.3
ERROR: Input tree has errors, aborting (use -f to force output)
scripts/Makefile.lib:285: recipe for target 'arch/arm/boot/dts/qcom-ipq8064-ea7500-v1.dtb' failed
make[6]: *** [arch/arm/boot/dts/qcom-ipq8064-ea7500-v1.dtb] Error 2
Makefile:1251: recipe for target 'dtbs' failed
make[5]: *** [dtbs] Error 2
make[5]: Leaving directory '/media/jack/sam128/Gmobile/openwrt-ipq806x-nss/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/linux-5.4.45'
Makefile:26: recipe for target '/media/jack/sam128/Gmobile/openwrt-ipq806x-nss/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/linux-5.4.45/.image' failed
make[4]: *** [/media/jack/sam128/Gmobile/openwrt-ipq806x-nss/build_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/linux-ipq806x_generic/linux-5.4.45/.image] Error 2
make[4]: Leaving directory '/media/jack/sam128/Gmobile/openwrt-ipq806x-nss/target/linux/ipq806x'
Makefile:13: recipe for target 'install' failed
make[3]: *** [install] Error 2
make[3]: Leaving directory '/media/jack/sam128/Gmobile/openwrt-ipq806x-nss/target/linux'
time: target/linux/install#57.43#28.90#93.40
target/Makefile:23: recipe for target 'target/linux/install' failed
make[2]: *** [target/linux/install] Error 2
make[2]: Leaving directory '/media/jack/sam128/Gmobile/openwrt-ipq806x-nss'
target/Makefile:19: recipe for target '/media/jack/sam128/Gmobile/openwrt-ipq806x-nss/staging_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/stamp/.target_install' failed
make[1]: *** [/media/jack/sam128/Gmobile/openwrt-ipq806x-nss/staging_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/stamp/.target_install] Error 2
make[1]: Leaving directory '/media/jack/sam128/Gmobile/openwrt-ipq806x-nss'
/media/jack/sam128/Gmobile/openwrt-ipq806x-nss/include/toplevel.mk:222: recipe for target 'world' failed
make: *** [world] Error 2

What I meant in my earlier post is that you should build an OpenWrt image without the NSS driver changes and make sure that image can run in your EA7500v1. Only then should you proceed with the NSS changes. Incorporating too many changes at once will make it very difficult to debug, especially with something as invasive as the NSS driver changes required.

From your post, I can't really tell if you're running someone else build or your own. The error log you posted does not seem related to the NSS changes that we've done.

Sorry,I didn't state clearly
I built the openwrt image for ea7500 v1 without no NSS drivers,it's openwrt offical
the error log is from building https://github.com/SqTER-PL/openwrt-ipq806x-nss/tree/master-qsdk10 for my ea7500v1 ,I guess if @Ansuel or @sqter updated the last master openwrt source code ,then my issue maybe will be fixed.

1 Like

Back for another round to try this but this time without :beer:

Merged again Ansuel kernel5.4-nss-qsdk10.0 branch on top of the recent master commit.

Summary
commit 02ca246ae57ff67fc8ed88e68a8466ca7990493d (HEAD -> master)
Merge: f632747704 118b4a3364
Author: gram <gram@Fedora-64.lan>
Date:   Fri Jul 10 16:53:29 2020 +0800

    Merge remote-tracking branch 'Ansuel/kernel5.4-nss-qsdk10.0'

commit f632747704f172e5c6d3a7c5715dc8d2f50d8da8 (origin/master, origin/HEAD)
Author: David Woodhouse <dwmw2@infradead.org>
Date:   Thu Jul 9 09:56:18 2020 +0100

    mediatek: fix bashism in gen_banana_pi_img.sh
    
    There was a bashism in the script. This fixes the script so that it
    doesn't actually require bash, and can be run with any POSIX shell as
    its shebang suggests.
    
    Signed-off-by: David Woodhouse <dwmw2@infradead.org>

But compilation fails when trying to apply 0069-arm-boot-add-dts-files.patch

Applying /home/gram/nss_test/openwrt/target/linux/ipq806x/patches-5.4/0069-arm-boot-add-dts-files.patch using plaintext: 
patching file arch/arm/boot/dts/Makefile
patch: **** malformed patch at line 35:  	qcom-msm8974-fairphone-fp2.dtb \

All the previous patches before 0069-arm-boot-add-dts-files.patch applies ok.

Is this patch supposed to apply cleanly or is my workflow incorrect and missing something or any other pointers why this patch fails?

unless you are trying to add some device you can restore the original add dts patch

Thanks Ansuel,

Compiles ok after copying the same dts patch from master.
Also booting ok this time.

But not offloading traffic to nss cores as CPU hits 100% during wired speedtest®.

NSS driver seems to be loaded/started:

[   13.621624] **********************************************************
[   13.621647] * Driver    :NSS GMAC Driver - RTL v(3.72a)
[   13.627033] * Version   :1.0
[   13.632259] * Copyright :Copyright (c) 2013-2018 The Linux Foundation. All rights reserved.
[   13.635284] **********************************************************
[   13.755308] nss_driver - fw of size 536164  bytes copied to load addr: 40000000, nss_id : 0
[   13.755996] nss_driver - Turbo Support 1
[   13.762852] Supported Frequencies - 
[   13.762855] 110Mhz 
[   13.766619] 600Mhz 
[   13.770248] 800Mhz 
[   13.771999] 
[   13.776430] fbc75041: meminfo init succeed
[   13.799497] nss_driver - fw of size 218096  bytes copied to load addr: 40800000, nss_id : 1
[   13.799756] 3dfe8968: meminfo init succeed
[   13.806706] debugfs: File 'n2h' in directory 'stats' already present!
[   13.808860] node size 1 # items 2
[   13.810908] debugfs: File 'qrfs' in directory 'stats' already present!
[   13.817339] memory: 0 0 (avl 487059456) items 2 active_cores 2
[   13.820675] debugfs: File 'c2c_tx' in directory 'stats' already present!
[   13.827048] fbc75041: nss core 0 booted successfully
[   13.832923] debugfs: File 'c2c_rx' in directory 'stats' already present!
[   13.844677] debugfs: File 'unaligned' in directory 'stats' already present!
[   13.854014] node size 1 # items 2
[   13.858038] memory: 0 0 (avl 487059456) items 2 active_cores 2
[   13.861568] 3dfe8968: nss core 1 booted successfully

Only below nss related selected from menuconfig:

< > kmod-qca-nss-cfi-cryptoapi..................... Kernel driver for NSS cfi
< > kmod-qca-nss-crypto.................. Kernel driver for NSS crypto driver
<*> kmod-qca-nss-drv..................... Kernel driver for NSS (core driver
<*> kmod-qca-nss-gmac............................. Kernel driver for NSS gmac

Logs full of below msgs:

Fri Jul 10 20:23:55 2020 kern.warn kernel: [  596.786817] net_ratelimit: 64 callbacks suppressed
Fri Jul 10 20:23:55 2020 kern.err kernel: [  596.786874] wlan0: NSS TX failed with error[8]: NSS_TX_FAILURE_NOT_ENABLED
Fri Jul 10 20:23:55 2020 kern.err kernel: [  596.886911] wlan0: NSS TX failed with error[8]: NSS_TX_FAILURE_NOT_ENABLED
Fri Jul 10 20:23:58 2020 kern.err kernel: [  599.641105] wlan0: NSS TX failed with error[8]: NSS_TX_FAILURE_NOT_ENABLED
Fri Jul 10 20:23:58 2020 kern.err kernel: [  599.943957] wlan0: NSS TX failed with error[8]: NSS_TX_FAILURE_NOT_ENABLED

What I'm missing from my config/setup to get traffic to be offloaded to nss cores?

I'm guessing this is ecm related as nothing is selected in menuconfig for it as nothing available for it.
How/what should be selected from menuconfig to get ecm frontend working?

think you miss to select ecm standard package should be in network support

Thanks,

Was guessing that already as saw it earlier under ppp, but are we referring to below highlighted?

Further assuming yes, then a nitpick why this is behind ppp ?
I'm most likely not the only one who is not using ppp so never selected it.
If ppp is not mandatory for ecm, I would suggest to remove the ecm standard package dependency from ppp or at least make it possible 'unselect' additional ppp packages after selecting the ecm standard package.
Now when selecting standard ecm it will come with couple extra ppp packages.

Let's see how it goes. I'll recompile and report back.
Anyhow a huge thanks to you, quarky & robimarko for making NSS offload possible.

:smiley:

Compiled/loaded ok after ecm standard package selection.

Have 1Gb up/down line from ISP and got +900 Mbps up and down with speedtest® with a modest max 20% CPU load by eyeballing htop at the same time.
Hurrah!

WIll monitor stability over the weekend whether returning back to standard master for the weekdays as WFH requires utmost stability.

Thanks guys, excellent!

While trying to build ansuel's changes merged into master for R7800 I am getting following questions:

PHY Core (GENERIC_PHY) [Y/?] y
Broadcom Kona USB2 PHY Driver (BCM_KONA_USB2_PHY) [N/m/y/?] n
Cadence MHDP DisplayPort PHY driver (PHY_CADENCE_DP) [N/m/y/?] n
Cadence D-PHY Support (PHY_CADENCE_DPHY) [N/m/y/?] n
Cadence Sierra PHY Driver (PHY_CADENCE_SIERRA) [N/m/y/?] n
Freescale i.MX8M USB3 PHY (PHY_FSL_IMX8MQ_USB) [N/m/y/?] n
Mixel MIPI DSI PHY support (PHY_MIXEL_MIPI_DPHY) [N/m/y/?] n
Marvell USB HSIC 28nm PHY Driver (PHY_PXA_28NM_HSIC) [N/m/y/?] n
Marvell USB 2.0 28nm PHY Driver (PHY_PXA_28NM_USB2) [N/m/y/?] n
Motorola Mapphone MDM6600 modem USB PHY driver (PHY_MAPPHONE_MDM6600) [N/m/y/?] 
n
SerDes PHY driver for Microsemi Ocelot (PHY_OCELOT_SERDES) [N/m/y/?] n
Qualcomm APQ8064 SATA SerDes/PHY driver (PHY_QCOM_APQ8064_SATA) [N/m/y/?] n
Qualcomm IPQ806x SATA SerDes/PHY driver (PHY_QCOM_IPQ806X_SATA) [Y/n/m/?] y
Qualcomm PCIe Gen2 PHY Driver (PHY_QCOM_PCIE2) [N/m/y/?] n
Qualcomm QMP PHY Driver (PHY_QCOM_QMP) [N/m/y/?] n
Qualcomm QUSB2 PHY Driver (PHY_QCOM_QUSB2) [N/m/y/?] n
Qualcomm UFS PHY driver (PHY_QCOM_UFS) [N/m/y/?] n
Qualcomm IPQ806x DWC3 USB PHY driver (PHY_QCOM_IPQ806X_USB) [N/m/y/?] (NEW) ESC[B
Qualcomm IPQ806x DWC3 USB PHY driver (PHY_QCOM_IPQ806X_USB) [N/m/y/?] (NEW) N

What are the right answers?

just click Y and press enter

I made some comparisons between Ansuel's NSS and hnyman's build:

			Ansuel		hnyman		Ansuel				Ansuel
									hw flow offloading	no soft offloading
iperf3		550 Mbps	910 Mbps	815 Mbps			610 Mbps
(wired)
iperf3		400 Mbps	700 Mbps	450-600 Mbps		480-640 Mbps
(wireless)
bufferbloat	B			A			B					C
(wired)
speedtest	77 Mbps		270 Mbps	100 Mbps			270 Mbps
(wireless)
uptime		few minutes	stable		> few minutes		> few minutes

Although low CPU utilization during tests involving routing looks promising stability and to lesser extend performance are still an issue.
Not sure whether difference in performance is because of lack of some patches and tweaks or if there is different reason.
Would it make sense to disable software flow offloading?

EDIT: in order to be able to turn on hardware flow offloading software flow offloading has to be checked. Table updated with Ansuel's repo with disabled software flow offloading.