Adding OpenWrt support for Xiaomi AX3600 (Part 1)

Flole · May 30, 2021, 6:35pm

Ah okay, then I need to check that again. There doesn't seem to be a perst-gpio in the stock DTS at all and in general that thing is a mess which is hard to understand.

Looking at your dts:

&pcie0 {
        status = "okay";

        perst-gpio = <&tlmm 52 GPIO_ACTIVE_HIGH>;

        bridge@0,0 {
                reg = <0x00000000 0 0 0 0>;
                #address-cells = <3>;
                #size-cells = <2>;
                ranges;

                wifi0: wifi@1,0 {
                        status = "okay";

                        compatible = "qcom,ath10k";
                        reg = <0x00010000 0 0 0 0>;

                        qcom,ath10k-calibration-variant = "Xiaomi-AX3600";
                };


        };
};

wifi0 is the PCI Radio? If I see that correctly mine is a ath11k based one, so that's probably one thing which needs changing.

Also I can't get the 2.5Gbit/s NIC to get a link (it is recognized now though).

Apache14 · May 30, 2021, 6:39pm

Ahh I thought you had found the fix for 5.13, if I can get anywhere further (or totally give up) on finding the memory leak I'll see if I can find anything in 5.13 for pcie

robimarko · May 30, 2021, 6:40pm

That DTS node for PCI radio is hardcoded for the ath10k radio.
You can remove the bridge@ node completely.
wifi0 is just a phandle, nothing else.

There has to the perst-gpio, maybe not directly called that but there is.

As far as 2.5G ports goes, it requrest the ess-switch, mdio phys and DP nodes to be changed.

robimarko · May 30, 2021, 6:40pm

No, unfortunately not.
I also checked on IPQ6010 and there it will hang for a minute and then finish PCI probe, but on IPQ807x it will just fail with CE copy error, engine poke etc errors.

@Ansuel Only ECM patches to kernel for offloading are the 999 core and UDP one?

Ansuel · May 30, 2021, 6:42pm

yes... anyway with some testing for some reason i got pci to randomly probe... isn't it strange that it does randomly probe? could be that we are missing some sleep between some reg apply ?

robimarko · May 30, 2021, 6:43pm

Ok, I will add those.

For PCI you are talking about 5.13 or OpenWrt?
Because in OpenWrt 5.10 it works.

Ansuel · May 30, 2021, 6:43pm

openwrt 5.10

robimarko · May 30, 2021, 6:44pm

Then you gotta be running some really old branch, PCI has been working fine for a while now.

Apache14 · May 30, 2021, 6:56pm

Btw did you remove my patch to read the number of clocks / resets from DTS ? If so that will probably cause issues with my uboot version.

robimarko · May 30, 2021, 6:58pm

Yes, it was never due to number of resets.
I found the root cause of the reset error, index for the last reset was reused.
Then getting the reset exclusively would fail as it was already reserved.
It was an error from applying a downstream clock patch.

U-boot should not matter at all as it was a driver issue.

Apache14 · May 30, 2021, 7:01pm

Ahh fantastic, in that case yeah my ath10k radio has been reliable

Flole · May 30, 2021, 7:02pm

My dts looks like this:

// SPDX-License-Identifier: GPL-2.0-or-later OR MIT

/dts-v1/;

#include "ipq8074.dtsi"
#include "ipq8074-ac-cpu.dtsi"
#include <dt-bindings/gpio/gpio.h>
#include <dt-bindings/input/input.h>

/ {
        #address-cells = <2>;
        #size-cells = <2>;

        model = "Netgear SXR80";
        compatible = "netgear,sxr80", "qcom,ipq8074";
        interrupt-parent = <&intc>;

        aliases {
                serial0 = &blsp1_uart5;
                led-boot = &led_front_blue;
                led-failsafe = &led_power_red;
                led-running = &led_power_green;
                led-upgrade = &led_front_white;
                /* Aliases as required by u-boot to patch MAC addresses */
                ethernet0 = "/soc/dp1";
                ethernet1 = "/soc/dp2";
                ethernet2 = "/soc/dp3";
                ethernet3 = "/soc/dp4";
                ethernet4 = "/soc/dp5";
        };

        chosen {
                stdout-path = "serial0:115200n8";
                bootargs-append = " root=/dev/ubiblock0_1";
        };

        keys {
                compatible = "gpio-keys";

                reset {
                        label = "reset";
                        gpios = <&tlmm 54 GPIO_ACTIVE_LOW>;
                        linux,code = <KEY_RESTART>;
                };

                wps {
                        label = "wps";
                        gpios = <&tlmm 57 GPIO_ACTIVE_LOW>;
                        linux,code = <KEY_WPS_BUTTON>;
                };
        };

        leds {
                compatible = "gpio-leds";

                led_front_blue: front-blue {
                        label = "blue:front";
                        gpios = <&tlmm 33 GPIO_ACTIVE_LOW>;
                };

                led_front_green: front-green {
                        label = "green:front";
                        gpios = <&tlmm 29 GPIO_ACTIVE_LOW>;
                };

                led_front_red: front-red {
                        label = "red:front";
                        gpios = <&tlmm 31 GPIO_ACTIVE_LOW>;
                };

                led_front_white: front-white {
                        label = "white:front";
                        gpios = <&tlmm 26 GPIO_ACTIVE_LOW>;
                };

                led_power_green: power-green {
                        label = "green:power";
                        gpios = <&tlmm 21 GPIO_ACTIVE_LOW>;
                };

                led_power_red: power-red {
                        label = "red:power";
                        gpios = <&tlmm 22 GPIO_ACTIVE_LOW>;
                };

        };
};

&tlmm {
        mdio_pins: mdio-pins {
                mdc {
                        pins = "gpio68";
                        function = "mdc";
                        drive-strength = <8>;
                        bias-pull-up;
                };

                mdio {
                        pins = "gpio69";
                        function = "mdio";
                        drive-strength = <8>;
                        bias-pull-up;
                };
        };

        leds_pins: leds_pinmux {
                led_power_green {
                        pins = "gpio21";
                        function = "gpio";
                        drive-strength = <8>;
                        bias-pull-down;
                };
                led_power_red {
                        pins = "gpio22";
                        function = "gpio";
                        drive-strength = <8>;
                        bias-pull-down;
                };
                led_white {
                        pins = "gpio26";
                        function = "gpio";
                        drive-strength = <8>;
                        bias-pull-down;
                };
                led_green {
                        pins = "gpio29";
                        function = "gpio";
                        drive-strength = <8>;
                        bias-pull-down;
                };
                led_red {
                        pins = "gpio31";
                        function = "gpio";
                        drive-strength = <8>;
                        bias-pull-down;
                };
                led_blue {
                        pins = "gpio33";
                        function = "gpio";
                        drive-strength = <8>;
                        bias-pull-down;
                };
        };


};

&blsp1_uart5 {
        status = "okay";
};

&prng {
        status = "okay";
};

&cryptobam {
        status = "okay";
};

&crypto {
        status = "okay";
};

&qpic_bam {
        status = "okay";
};

&qpic_nand {
        status = "okay";

        nand@0 {
                reg = <0>;
                nand-ecc-strength = <4>;
                nand-ecc-step-size = <512>;
                nand-bus-width = <8>;

                partitions {
                        compatible = "qcom,smem-part";
                };
        };
};

&mdio {
        status = "okay";

        pinctrl-0 = <&mdio_pins>;
        pinctrl-names = "default";
        phy-reset-gpio = <&tlmm 37 0 &tlmm 25 1>;

        ethernet-phy@1 {
                reg = <1>;
        };

        ethernet-phy@2 {
                reg = <2>;
        };

        ethernet-phy@3 {
                reg = <3>;
        };

        ethernet-phy@4 {
                reg = <4>;
        };

        ethernet-phy@5 {
                reg = <28>;
        };

};

&qmp_pcie_phy0 {
        status = "okay";
};

&pcie0 {
        status = "okay";

        perst-gpio = <&tlmm 58 1>;

};

&pcie1 {
        status = "okay";

        perst-gpio = <&tlmm 61 1>;

};

&ess_switch {
        switch_cpu_bmp = <0x1>;  /* cpu port bitmap */
        switch_lan_bmp = <0x3e>; /* lan port bitmap */
        switch_wan_bmp = <0x40>; /* wan port bitmap */
        switch_mac_mode = <0x0>; /* mac mode for uniphy instance0*/
        switch_mac_mode1 = <0xff>; /* mac mode for uniphy instance1*/
        switch_mac_mode2 = <0xf>; /* mac mode for uniphy instance2*/
        bm_tick_mode = <0>; /* bm tick mode */
        tm_tick_mode = <0>; /* tm tick mode */
        qcom,port_phyinfo {
                port@1 {
                        port_id = <2>;
                        phy_address = <1>;
                };
                port@2 {
                        port_id = <3>;
                        phy_address = <2>;
                };
                port@3 {
                        port_id = <4>;
                        phy_address = <3>;
                };
                port@4 {
                        port_id = <5>;
                        phy_address = <4>;
                };
                port@5 {
                        port_id = <6>;
                        phy_address = <28>;
                        port_mac_sel = "QGMAC_PORT";
                };
        };
        port_scheduler_resource {
                port@0 {
                        port_id = <0>;
                        ucast_queue = <0 143>;
                        mcast_queue = <256 271>;
                        l0sp = <0 35>;
                        l0cdrr = <0 47>;
                        l0edrr = <0 47>;
                        l1cdrr = <0 7>;
                        l1edrr = <0 7>;
                };
                port@1 {
                        port_id = <1>;
                        ucast_queue = <144 159>;
                        mcast_queue = <272 275>;
                        l0sp = <36 39>;
                        l0cdrr = <48 63>;
                        l0edrr = <48 63>;
                        l1cdrr = <8 11>;
                        l1edrr = <8 11>;
                };
                port@2 {
                        port_id = <2>;
                        ucast_queue = <160 175>;
                        mcast_queue = <276 279>;
                        l0sp = <40 43>;
                        l0cdrr = <64 79>;
                        l0edrr = <64 79>;
                        l1cdrr = <12 15>;
                        l1edrr = <12 15>;
                };
                port@3 {
                        port_id = <3>;
                        ucast_queue = <176 191>;
                        mcast_queue = <280 283>;
                        l0sp = <44 47>;
                        l0cdrr = <80 95>;
                        l0edrr = <80 95>;
                        l1cdrr = <16 19>;
                        l1edrr = <16 19>;
                };
                port@4 {
                        port_id = <4>;
                        ucast_queue = <192 207>;
                        mcast_queue = <284 287>;
                        l0sp = <48 51>;
                        l0cdrr = <96 111>;
                        l0edrr = <96 111>;
                        l1cdrr = <20 23>;
                        l1edrr = <20 23>;
                };
                port@5 {
                        port_id = <5>;
                        ucast_queue = <208 223>;
                        mcast_queue = <288 291>;
                        l0sp = <52 55>;
                        l0cdrr = <112 127>;
                        l0edrr = <112 127>;
                        l1cdrr = <24 27>;
                        l1edrr = <24 27>;
                };
                port@6 {
                        port_id = <6>;
                        ucast_queue = <224 239>;
                        mcast_queue = <292 295>;
                        l0sp = <56 59>;
                        l0cdrr = <128 143>;
                        l0edrr = <128 143>;
                        l1cdrr = <28 31>;
                        l1edrr = <28 31>;
                };
                port@7 {
                        port_id = <7>;
                        ucast_queue = <240 255>;
                        mcast_queue = <296 299>;
                        l0sp = <60 63>;
                        l0cdrr = <144 159>;
                        l0edrr = <144 159>;
                        l1cdrr = <32 35>;
                        l1edrr = <32 35>;
                };
        };
        port_scheduler_config {
                port@0 {
                        port_id = <0>;
                        l1scheduler {
                                group@0 {
                                        sp = <0 1>; /*L0 SPs*/
                                        /*cpri cdrr epri edrr*/
                                        cfg = <0 0 0 0>;
                                };
                        };
                        l0scheduler {
                                group@0 {
                                        /*unicast queues*/
                                        ucast_queue = <0 4 8>;
                                        /*multicast queues*/
                                        mcast_queue = <256 260>;
                                        /*sp cpri cdrr epri edrr*/
                                        cfg = <0 0 0 0 0>;
                                };
                                group@1 {
                                        ucast_queue = <1 5 9>;
                                        mcast_queue = <257 261>;
                                        cfg = <0 1 1 1 1>;
                                };
                                group@2 {
                                        ucast_queue = <2 6 10>;
                                        mcast_queue = <258 262>;
                                        cfg = <0 2 2 2 2>;
                                };
                                group@3 {
                                        ucast_queue = <3 7 11>;
                                        mcast_queue = <259 263>;
                                        cfg = <0 3 3 3 3>;
                                };
                        };
                };
                port@1 {
                        port_id = <1>;
                        l1scheduler {
                                group@0 {
                                        sp = <36>;
                                        cfg = <0 8 0 8>;
                                };
                                group@1 {
                                        sp = <37>;
                                        cfg = <1 9 1 9>;
                                };
                        };
                        l0scheduler {
                                group@0 {
                                        ucast_queue = <144>;
                                        ucast_loop_pri = <16>;
                                        mcast_queue = <272>;
                                        mcast_loop_pri = <4>;
                                        cfg = <36 0 48 0 48>;
                                };
                        };
                };
                port@2 {
                        port_id = <2>;
                        l1scheduler {
                                group@0 {
                                        sp = <40>;
                                        cfg = <0 12 0 12>;
                                };
                                group@1 {
                                        sp = <41>;
                                        cfg = <1 13 1 13>;
                                };
                        };
                        l0scheduler {
                                group@0 {
                                        ucast_queue = <160>;
                                        ucast_loop_pri = <16>;
                                        mcast_queue = <276>;
                                        mcast_loop_pri = <4>;
                                        cfg = <40 0 64 0 64>;
                                };
                        };
                };
                port@3 {
                        port_id = <3>;
                        l1scheduler {
                                group@0 {
                                        sp = <44>;
                                        cfg = <0 16 0 16>;
                                };
                                group@1 {
                                        sp = <45>;
                                        cfg = <1 17 1 17>;
                                };
                        };
                        l0scheduler {
                                group@0 {
                                        ucast_queue = <176>;
                                        ucast_loop_pri = <16>;
                                        mcast_queue = <280>;
                                        mcast_loop_pri = <4>;
                                        cfg = <44 0 80 0 80>;
                                };
                        };
                };
                port@4 {
                        port_id = <4>;
                        l1scheduler {
                                group@0 {
                                        sp = <48>;
                                        cfg = <0 20 0 20>;
                                };
                                group@1 {
                                        sp = <49>;
                                        cfg = <1 21 1 21>;
                                };
                        };
                        l0scheduler {
                                group@0 {
                                        ucast_queue = <192>;
                                        ucast_loop_pri = <16>;
                                        mcast_queue = <284>;
                                        mcast_loop_pri = <4>;
                                        cfg = <48 0 96 0 96>;
                                };
                        };
                };
                port@5 {
                        port_id = <5>;
                        l1scheduler {
                                group@0 {
                                        sp = <52>;
                                        cfg = <0 24 0 24>;
                                };
                                group@1 {
                                        sp = <53>;
                                        cfg = <1 25 1 25>;
                                };
                        };
                        l0scheduler {
                                group@0 {
                                        ucast_queue = <208>;
                                        ucast_loop_pri = <16>;
                                        mcast_queue = <288>;
                                        mcast_loop_pri = <4>;
                                        cfg = <52 0 112 0 112>;
                                };
                        };
                };
                port@6 {
                        port_id = <6>;
                        l1scheduler {
                                group@0 {
                                        sp = <56>;
                                        cfg = <0 28 0 28>;
                                };
                                group@1 {
                                        sp = <57>;
                                        cfg = <1 29 1 29>;
                                };
                        };
                        l0scheduler {
                                group@0 {
                                        ucast_queue = <224>;
                                        ucast_loop_pri = <16>;
                                        mcast_queue = <292>;
                                        mcast_loop_pri = <4>;
                                        cfg = <56 0 128 0 128>;
                                };
                        };
                };
                port@7 {
                        port_id = <7>;
                        l1scheduler {
                                group@0 {
                                        sp = <60>;
                                        cfg = <0 32 0 32>;
                                };
                                group@1 {
                                        sp = <61>;
                                        cfg = <1 33 1 33>;
                                };
                        };
                        l0scheduler {
                                group@0 {
                                        ucast_queue = <240>;
                                        ucast_loop_pri = <16>;
                                        mcast_queue = <296>;
                                        cfg = <60 0 144 0 144>;
                                };
                        };
                };
        };
};

&soc {
        dp1: dp1 {
                device_type = "network";
                compatible = "qcom,nss-dp";
                qcom,id = <1>;
                reg = <0x3a001000 0x200>;
                qcom,mactype = <0>;
                local-mac-address = [000000000000];
                qcom,link-poll = <1>;
                qcom,phy-mdio-addr = <0>;
                phy-mode = "sgmii";
                mdio-bus = <&mdio>;
        };

        dp2: dp2 {
                device_type = "network";
                compatible = "qcom,nss-dp";
                qcom,id = <2>;
                reg = <0x3a001200 0x200>;
                qcom,mactype = <0>;
                local-mac-address = [000000000000];
                qcom,link-poll = <1>;
                qcom,phy-mdio-addr = <1>;
                phy-mode = "sgmii";
                mdio-bus = <&mdio>;
        };

        dp3: dp3 {
                device_type = "network";
                compatible = "qcom,nss-dp";
                qcom,id = <3>;
                reg = <0x3a001400 0x200>;
                qcom,mactype = <0>;
                local-mac-address = [000000000000];
                qcom,link-poll = <1>;
                qcom,phy-mdio-addr = <2>;
                phy-mode = "sgmii";
                mdio-bus = <&mdio>;
        };

        dp4: dp4 {
                device_type = "network";
                compatible = "qcom,nss-dp";
                qcom,id = <4>;
                reg = <0x3a001600 0x200>;
                qcom,mactype = <0>;
                local-mac-address = [000000000000];
                qcom,link-poll = <1>;
                qcom,phy-mdio-addr = <3>;
                phy-mode = "sgmii";
                mdio-bus = <&mdio>;
        };

        dp5: dp5 {
                device_type = "network";
                compatible = "qcom,nss-dp";
                qcom,id = <5>;
                reg = <0x3a001800 0x200>;
                qcom,mactype = <0>;
                local-mac-address = [000000000000];
                qcom,link-poll = <1>;
                qcom,phy-mdio-addr = <4>;
                phy-mode = "sgmii";
                mdio-bus = <&mdio>;
        };

};

&wifi {
        status = "okay";

        qcom,board_id = <658>;
        qcom,ath11k-calibration-variant = "Xiaomi-AX3600";
};

I think I have done all the required changes and it does show up as an eth (it obviously didn't without the changes) so I am not sure what I might be missing. I had an additional one in there which I removed now, not sure if that caused an issue. It's recompiling now again (I guess there is no way to do a faster recompile if I just changed the dts?)

I found some perst-gpio settings and setup pcie0 and pcie1 now, let's see what happens.

Also they are using the TLC59208F i2c LED Driver (despite having the GPIOs declared in the dts and in uboot, but they simply don't work and are never loaded in the original firmware as there is a collision between the LEDs and I think some sound stuff). Should I even try to get that to work or what was your approach for the other Netgear devices which had the LEDs on the i2c?

robimarko · May 30, 2021, 7:09pm

Can you post the stock DTS as well?

BTW TLC59208F was used in the SRK60 I had, it's supported in Linux.
Take a look at the Orbi SRK60 in OpenWrt, DTS for it is pretty clear, but you also need to enable the driver as well.
Also, the I2C bus needs the correct one to which the LED driver is connected.

Sorry, but networking nodes will not be correct for your board.

I am sorry, but can you just wait until the AX9000 arrives and we have a solid base?
I mean, we are still working on various stuff on AX3600 as well.

@Ansuel I pushed the ECM to the NSS repo and added the 2 patches to the AX3600 tree.
Would it be possible to break those 2 patches into more with commit messages?

Also, I noticed that offloading works when traffic originates from one of the ports, but not when it's from the AX3600 itself.
Would it also be possible to add VLAN and bridge offloading?
Those should be relatively simple.

@Apache14 Can you contribute PPoE offloading?

Ansuel · May 30, 2021, 7:11pm

bridge and vlan is disabled for ecm am I wrong?
could also be that ecm is detecting all the traffic from the bridge as local and is rejecting them (local traffic is not offloaded)

Apache14 · May 30, 2021, 7:12pm

Yeah I'll send over the PPPoE kernel patch tomorrow.

I'll also push the 11.3 based packages I have to the feed repo.

Vlan and bridge probably need to have the nss clients also built

https://source.codeaurora.org/quic/qsdk/oss/lklm/nss-clients/tree/vlan?h=NHSS.QSDK.11.3.0.6.r2

https://source.codeaurora.org/quic/qsdk/oss/lklm/nss-clients/tree/bridge?h=NHSS.QSDK.11.3.0.6.r2

robimarko · May 30, 2021, 7:15pm

@Ansuel VLAN is disabled for sure in the Makefile, bridge is not explicitly disabled.
No idea, but should for example iperf3 traffic from the device itself also be NAT-ed and offloaded.
Otherwise WLAN might not be offloaded as well if it only offloads physical ports and not bridged ones as well.

@Apache14 Thanks, yeah feel free to make a new branch on the repos and push.

Apache14 · May 30, 2021, 7:16pm

See above for either we will need the nss-client module for some of the hooks.

I can investigate both later next week

Ansuel · May 30, 2021, 7:17pm

vlan didn't change nss implementation for the drv... will check if ecm compiles correctly...

@robimarko can you do a quick test about bridge?

check the last commit try to revert the dst->flags change

Flole · May 30, 2021, 7:23pm

The stock DTS is kinda a mess, I took the networking stuff from there. It's to large to be posted here, so: https://pastebin.com/0JiVF9Qn

I think i2c 0 is the correct one, I'll try to get that thing to work as I probably won't need help for that if I use the SRK60 as template.

Apache14 · May 30, 2021, 7:23pm

You want the kernel patches on the nss-1.4 branch ?