Adding OpenWrt support for Extreme Network AP3805i

Dragets · December 27, 2020, 7:35pm

I've been working on this AP-device for a little while, trying to port OpenWrt to it. But now i thought i could use some help understanding the last bits and pieces. So far i've only been doing testing with initramfs-builds and not any flashing since the unit seem to be fully capable with TFTP-boot.

The device in question:
Extreme Networks WS-AP3805i.

SoC: Qualcomm Atheros QCA9557-AT4A
RAM: 2x 128MB Nanya NT5TU64M16HG
FLASH: 64MB - SPANSION FL512SAIFG1
LAN: Atheros AR8035-A switch (RGMII GbE with PoE+ IN)
WLAN2: Qualcomm Atheros QCA9557 2x2 2T2R
WLAN5: Qualcomm Atheros QCA9882-BR4A 2x2 2T2R
SERIAL: UART pins at J10 (115200 8n1)

Some of the more important files/logs extracted from the device:

rcs file: https://pastebin.com/J7CexJH8
rc.wlan file: https://pastebin.com/g2wSdK7m
environ file: https://pastebin.com/VcebmjMP
startup syslog: https://pastebin.com/vTaE566D
and the OEM bootlog: https://pastebin.com/g5xr0a2g

I've also dumped all the MTD-partitions to .bin files to do some digging.

From what i've come up to as of yet is the following DTS:

The following addition to the generic.mk:

define Device/extreme_ap3805i
  SOC := qca9557
  DEVICE_VENDOR := Extreme Systems
  DEVICE_MODEL := AP3805i
  DEVICE_PACKAGES := kmod-ath10k-ct ath10k-firmware-qca988x-ct
  IMAGE_SIZE := 32768k
endef
TARGET_DEVICES += extreme_ap3805i

I've also looked at Enterasys WS-AP3705i and Siemens AP3610 to get a start and do the ath9/10k fixes and network MAC fix.
The Cal-data on this device is not stored in flash. It has no ART/calibrate-partition and in the rc.wlan script the following is said about this board (db12x_p1):

            if [ "${BOARD_TYPE}" == "db12x_p1" ]; then
                mknod /dev/caldata c 63 4
                mknod /dev/caldata_pci c 63 5
                echo "*** Calibration data is stored in flash" > /dev/console
                echo "*** Please USE memory=flash in cart load command" > /dev/console
            else
                CALIBRATE=`cat /proc/mtd|grep "calibrate"`
                if [ "${CALIBRATE}" != "" ]; then
               ## Calibration data is stored in flash

                    flash_minor=`cat /proc/mtd|grep '"calibrate"'|cut -f1 -d":"|cut -c4-65535`
                    mknod /dev/caldata b 31 $flash_minor

                    echo "*** Calibration data is stored in flash" > /dev/console
                    echo "*** Please USE memory=flash in cart load command" > /dev/console
                else
                    echo "*** Calibration data is stored in ether EEPROM or OTP" > /dev/console
                    echo "*** Please DO NOT use memory=flash in cart load command" > /dev/console
                fi
            fi

If i understand it right, if i convert one of the NVRAM-partitions to a "calibrate"-partition i can use the drivers to dump the onboard CAL-data to that partition?

To the real problem tho:

Whatever i do, and whatever i change the device is rebooting at 30 seconds. I started thinking that it must be a GPIO-watchdog? The normal QCA register for the watchdog is all 00-ed out and even when compiling without any watchdog support at all the same thing repeats itself.
If i do a "md.b"-command with a register that makes U-boot look, the same timer seem to kick in. After 30seconds the unit restarts.

OpenWrt bootlog:

And lastly some pictures of the device:

Thank you for your time,

mpratt14 · December 28, 2020, 12:46am

Your DTS looks ok so far....
however the DTSI file needs to be first thing after license line, before the bindings header files. I don't know how important this is but this is the convention right now.

That script tells you nothing about the cal data unless you know exactly how the BOARD_TYPE variable is used.

If the cal data is truly on a physical EEPROM thats part of the wifi interface, the drivers will detect it and use it automatically so long as the qca,no-eeprom property is NOT in the DTS.

you have to actually look for the cal data on the flash and find out yourself whether its there or not, a good way to do this is grep a hexdump for one of the wireless MACs
hexdump -C /dev/mtdX | grep 'aa bb cc'

you should have looked at the existing partition table from the original software with
cat /proc/mtd
if you are able to get root shell access...
I would expect to see a partition called 'art'
but the absence of that can be an indicator that it is programmed into the wifi chip

Does this happen if you are in uboot console and doing nothing?

To me it sounds like a stuck reset button...or it is actually active HIGH instead of active LOW...you can try removing the reset button node for now maybe...

I have added 2 nearly identical boards, I hope studying these commits helps you (you can ignore the node called virtual_flash and the factory.bin recipe which is a unique situation)

github.com/openwrt/openwrt

ath79: add support for Senao Engenius ECB1200

committed 06:11PM - 22 Dec 20 UTC

mpratt14

+158 -0

FCC ID: A8J-ECB1200 Engenius ECB1200 is an indoor wireless access point with a …GbE port, 2.4 GHz and 5 GHz wireless, external antennas, and 802.3af PoE. **Specification:** - QCA9557 SOC MIPS, 2.4 GHz (2x2) - QCA9882 WLAN PCIe card, 5 GHz (2x2) - AR8035-A switch RGMII, GbE with 802.3af PoE, 25 MHz clock - 40 MHz reference clock - 16 MB FLASH 25L12845EMI-10G - 2x 64 MB RAM 1538ZFZ V59C1512164QEJ25 - UART at JP1 (unpopulated, RX shorted to ground) - 4 external antennas - 4 LEDs, 1 button (power, eth, wifi2g, wifi5g) (reset) **MAC addresses:** MAC Addresses are labeled as ETH and 5GHZ U-boot environment has the vendor MAC addresses MAC addresses in ART do not match vendor eth0 ETH *:5c u-boot-env ethaddr phy0 5GHZ *:5d u-boot-env athaddr ---- ---- ???? art 0x0/0x6 **Installation:** Method 1: Firmware upgrade page: OEM webpage at 192.168.1.1 username and password "admin" Navigate to "Firmware" page from left pane Click Browse and select the factory.bin image Upload and verify checksum Click Continue to confirm and wait 3 minutes Method 2: Serial to load Failsafe webpage: After connecting to serial console and rebooting... Interrupt uboot with any key pressed rapidly (see TFTP recovery) perform a sysupgrade **Serial Access:** the RX line on the board for UART is shorted to ground by resistor R176 therefore it must be removed to use the console but it is not necessary to remove to view boot log optionally, R175 can be replaced with a solder bridge short the resistors R175 and R176 are next to the UART pinout at JP1 **Return to OEM:** If you have a serial cable, see Serial Failsafe instructions Unlike most Engenius boards, this does not have a 'failsafe' image the only way to return to OEM is TFTP or serial access to u-boot **TFTP recovery:** Unlike most Engenius boards, TFTP is reliable here rename initramfs-kernel.bin to 'ap.bin' make the file available on a TFTP server at 192.168.1.10 power board while holding or pressing reset button repeatedly or with serial access: run `tftpboot` or `run factory_boot` with initramfs-kernel.bin then `bootm` with the load address **Format of OEM firmware image:** The OEM software of ECB1200 is a heavily modified version of Openwrt Altitude Adjustment 12.09. This Engenius board, like ECB1750, uses a proprietary header with a unique Product ID. The header for factory.bin is generated by the mksenaofw program included in openwrt. **Note on PLL-data cells:** The default PLL register values will not work because of the AR8035 switch between the SOC and the ethernet port. For QCA955x series, the PLL registers for eth0 and eth1 can be see in the DTSI as 0x28 and 0x48 respectively. Therefore the PLL registers can be read from uboot for each link speed after attempting tftpboot or another network action using that link speed with `md 0x18050028 1` and `md 0x18050048 1`. However the registers that u-boot sets are not ideal and sometimes wrong... the at803x driver supports setting the RGMII clock/data delay on the PHY side. This way the pll-data register only needs to handle invert and phase. for this board clock invert is needed on the MAC side all link speeds functional Signed-off-by: Michael Pratt <mcpratt@pm.me>

github.com/openwrt/openwrt

ath79: add support for Senao Engenius EnStationAC v1

committed 06:11PM - 22 Dec 20 UTC

mpratt14

+217 -1

FCC ID: A8J-ENSTAC Engenius EnStationAC v1 is an outdoor wireless access point/…bridge with 2 gigabit ethernet ports on 2 external ethernet switches, 5 GHz only wireless, internal antenna plates, and proprietery PoE. Specification: - QCA9557 SOC - QCA9882 WLAN (PCI card, 5 GHz, 2x2, 26dBm) - AR8035-A switch (RGMII GbE with PoE+ IN) - AR8031 switch (SGMII GbE with PoE OUT) - 40 MHz reference clock - 16 MB FLASH MX25L12845EMI-10G - 2x 64 MB RAM NT5TU32M16FG - UART at J10 (unpopulated) - internal antenna plates (19 dbi, directional) - 7 LEDs, 1 button (power, eth, wlan, RSSI) (reset) MAC addresses: MAC addresses are labeled as ETH and 5GHz Vendor MAC addresses in flash are duplicate eth0 ETH *:d3 art 0x0/0x6 eth1 ---- *:d4 --- phy0 5GHz *:d5 --- Installation: 2 ways to flash factory.bin from OEM: - if you get Failsafe Mode from failed flash: only use it to flash Original firmware from Engenius or risk kernel loop or halt which requires serial cable Method 1: Firmware upgrade page: OEM webpage at 192.168.1.1 username and password "admin" Navigate to "Firmware" page from left pane Click Browse and select the factory.bin image Upload and verify checksum Click Continue to confirm and wait 3 minutes Method 2: Serial to load Failsafe webpage: After connecting to serial console and rebooting... Interrupt uboot with any key pressed rapidly execute `run failsafe_boot` OR `bootm 0x9fd70000` wait a minute connect to ethernet and navigate to "192.168.1.1/index.htm" Select the factory.bin image and upload wait about 3 minutes Return to OEM: If you have a serial cable, see Serial Failsafe instructions otherwise, uboot-env can be used to make uboot load the failsafe image *DISCLAIMER* The Failsafe image is unique to Engenius boards. If the failsafe image is missing or damaged this will not work DO NOT downgrade to ar71xx this way, it can cause kernel loop or halt ssh into openwrt and run `fw_setenv rootfs_checksum 0` reboot, wait 3 minutes connect to ethernet and navigate to 192.168.1.1/index.htm select OEM firmware image from Engenius and click upgrade TFTP recovery: rename initramfs to 'vmlinux-art-ramdisk' make available on TFTP server at 192.168.1.101 power board hold or press reset button repeatedly NOTE: for some Engenius boards TFTP is not reliable try setting MTU to 600 and try many times Format of OEM firmware image: The OEM software of EnStationAC is a heavily modified version of Openwrt Altitude Adjustment 12.09. One of the many modifications is to the sysupgrade program. Image verification is performed simply by the successful ungzip and untar of the supplied file and name check and header verification of the resulting contents. To form a factory.bin that is accepted by OEM Openwrt build, the kernel and rootfs must have specific names... openwrt-ar71xx-enstationac-uImage-lzma.bin openwrt-ar71xx-enstationac-root.squashfs and begin with the respective headers (uImage, squashfs). Then the files must be tarballed and gzipped. The resulting binary is actually a tar.gz file in disguise. This can be verified by using binwalk on the OEM firmware images, ungzipping then untaring. Newer EnGenius software requires more checks but their script includes a way to skip them, otherwise the tar must include a text file with the version and md5sums in a deprecated format. The OEM upgrade script is at /etc/fwupgrade.sh. OKLI kernel loader is required because the OEM software expects the kernel to be no greater than 1536k and the factory.bin upgrade procedure would otherwise overwrite part of the kernel when writing rootfs. Note on PLL-data cells: The default PLL register values will not work because of the external AR8033 switch between the SOC and the ethernet PHY chips. For QCA955x series, the PLL registers for eth0 and eth1 can be see in the DTSI as 0x28 and 0x48 respectively. Therefore the PLL registers can be read from uboot for each link speed after attempting tftpboot or another network action using that link speed with `md 0x18050028 1` and `md 0x18050048 1`. For eth0 at 1000 speed, the value returned was ae000000 but that didn't work, so following the logical pattern from the rest of the values, the guessed value of a3000000 works better. later discovered that delay can be placed on the PHY end only with phy-mode as 'rgmii-id' and set register to 0x82... Tested from master, all link speeds functional Signed-off-by: Michael Pratt <mcpratt@pm.me> [fixed SoB to match From:] Signed-off-by: Petr Štetiar <ynezz@true.cz>

Dragets · December 28, 2020, 7:51am

Thanks for the input!

I've looked at the engenius-files to get some ideas as i understand this is a Senao-made one with Extreme Systems/Enterasys own small changes.

The device do not have any calibration or art-partition and i have searched through the dumped partition with no results.
The oem syslog gives this at boot:

krn:  scn->cal_mem a3fc4000 AH_CAL_LOCATIONS_PCI 3fc4000 HOST_CALDATA_SIZE 4000

And it can be found with md.b 3fc4000 after one oem-boot, then a restart to uboot. So i guess it actually is in the eeprom/otp?

I have tried compile without any keys, and even without any leds. But still restart at 30s mark.
Uboot console can be up for days without any restart. But if i do a command that makes it freeze, there is some kind of watchdog that restarts the unit in 30s.

mpratt14 · December 28, 2020, 6:22pm

How about failsafe mode when booting openwrt?

Dragets · December 28, 2020, 7:09pm

Same thing there, reboot after 30sec.
When examining the running oem-system i found this in "proc/simple_config"

Factory# ls /proc/simple_config
external_wdt_debug     flash_addressing_mode  radio1_led
external_wdt_enable    push_button            status_led
external_wdt_refresh   radio0_led
Factory# cat /proc/simple_config/external_wdt_debug
The external watchdog was last refreshed 18 secs ago
Refresh interval: 20 secs
Total times refreshed: 12

So i guess there is an external WDT triggering the reboot. Now i just need to find out where it is. Do you have any examples or tips i can use for finding it?

mpratt14 · December 28, 2020, 8:02pm

OK so at this point I would

start looking for a chip on the board that would be serving as the watchdog

if you have a multimeter and can identify the chip acting as a watchdog, see where it traces to the SOC (but this is difficult because it's BGA...)

(even better if you had an oscilloscope so you can see what type of signal is ideal)

analyze gpios with oem image....you can use gpio scripts from the wiki or looking for the individual gpio exports in /sys and compare that to the numbers you know to be LEDs and reset button

look for a script in oem that controls the watchdog...

look in /dev for something like "wdt" or "watchdog"

find / | grep wdt
find / | grep watchdog

I really don't know anything about this
but heres some reading material and an example device

https://www.ti.com/lit/an/slaa789/slaa789.pdf

mpratt14 · December 28, 2020, 8:13pm

If you find that it is indeed a GPIO pinout that "touches" the watchdog to keep the system alive, here is the kernel documenation for making a DTS node for that. You can call the node something like "ext-watchdog" and it would be within the root node (with aliases and keys and leds)

github.com

torvalds/linux/blob/master/Documentation/devicetree/bindings/watchdog/gpio-wdt.txt

* GPIO-controlled Watchdog

Required Properties:
- compatible: Should contain "linux,wdt-gpio".
- gpios: From common gpio binding; gpio connection to WDT reset pin.
- hw_algo: The algorithm used by the driver. Should be one of the
  following values:
  - toggle: Either a high-to-low or a low-to-high transition clears
    the WDT counter. The watchdog timer is disabled when GPIO is
    left floating or connected to a three-state buffer.
  - level: Low or high level starts counting WDT timeout,
    the opposite level disables the WDT. Active level is determined
    by the GPIO flags.
- hw_margin_ms: Maximum time to reset watchdog circuit (milliseconds).

Optional Properties:
- always-running: If the watchdog timer cannot be disabled, add this flag to
  have the driver keep toggling the signal without a client. It will only cease
  to toggle the signal when the device is open and the timeout elapsed.

This file has been truncated. show original

mpratt14 · December 28, 2020, 8:19pm

you can use lines from scripts here to help you test each GPIO

you should start at GPIO 12 because there are many between 0 and 12 that are used for other things, however it may be in GPIO 0 to 4

so you would boot openwrt and start toggling a gpio pin quickly to see if it stays alive or not

Dragets · December 29, 2020, 9:25pm

Thank you!

Could finaly figure it out with some digging through the GPIO-registers for QCA9558 (almost identical) in u-boot. Did see that GPIO16 was toggled every sec when in U-boot. So i did a test with a build with a GPIO16-watchdog and it works!

Now i can work on having the drivers right.

mpratt14 · December 29, 2020, 10:48pm

good job, if you need help creating a pull request let me know

mpratt14 · December 30, 2020, 9:21pm

BTW here is an example with something already in openwrt master

probably you want a longer toggle time like 2 seconds

github.com

openwrt/openwrt/blob/master/target/linux/ath79/dts/qca9558_comfast_cf-wr650ac.dtsi#L45


			gpios = <&gpio 13 GPIO_ACTIVE_LOW>;
			linux,default-trigger = "phy1tpt";
		};
		wlan5g {
			label = "blue:wlan5";
			gpios = <&gpio 2 GPIO_ACTIVE_LOW>;
			linux,default-trigger = "phy0tpt";
		};
	};
	watchdog {
		compatible = "linux,wdt-gpio";
		gpios = <&gpio 17 GPIO_ACTIVE_LOW>;
		hw_algo = "toggle";
		hw_margin_ms = <500>;
		always-running;
	};
};
&pcie0 {

Dragets · January 3, 2021, 10:47pm

Ok, so this is where i'm at now after the holidays are over and i can put some more time into this.

My DTS is now looking like this:

// SPDX-License-Identifier: GPL-2.0-or-later OR MIT

#include "qca955x.dtsi"

#include <dt-bindings/gpio/gpio.h>
#include <dt-bindings/input/input.h>

/ {
	compatible = "extreme,ap3805i", "qca,qca9557";
	model = "Extreme Networks AP3805i";

	aliases {
		led-boot = &led_power_green;
		led-failsafe = &led_power_red;
		led-running = &led_power_green;
		led-upgrade = &led_power_red;
		label-mac-device = &eth0;
	};

	leds {
		compatible = "gpio-leds";

		led_power_green: led_power_green {
			label = "ap3805:green:power";
			gpios = <&gpio 19 GPIO_ACTIVE_LOW>;
		};

		led_power_red: led_power_red {
			label = "ap3805:red:power";
			gpios = <&gpio 23 GPIO_ACTIVE_LOW>;
		};

		led_wlan5_green {
			label = "ap3805:green:wlan5";
			gpios = <&gpio 13 GPIO_ACTIVE_LOW>;
		};	

		led_wlan2_green {
			label = "ap3805:green:wlan2";
			gpios = <&gpio 22 GPIO_ACTIVE_LOW>;
		};

	};

	keys {
		compatible = "gpio-keys";

		reset {
			label = "reset";
			linux,code = <KEY_RESTART>;
			gpios = <&gpio 17 GPIO_ACTIVE_LOW>;
		};
	};

	watchdog {
		compatible = "linux,wdt-gpio";
		gpios = <&gpio 16 GPIO_ACTIVE_LOW>;
		hw_algo = "toggle";
		hw_margin_ms = <20000>;
		always-running;
	};
};

&uart {
	status = "okay";
};

&wmac {
	status = "okay";
};

&pcie0 {
	status = "okay";
};

&mdio0 {
	status = "okay";

	phy5: ethernet-phy@5 {
		reg = <5>;
	};
};

&eth0 {
	status = "okay";

	phy-handle = <&phy5>;
};

&spi {
	status = "okay";

	num-cs = <1>;
	
	flash@0 {
		compatible = "jedec,spi-nor";
		reg = <0>;
		spi-max-frequency = <80000000>;

		partitions {
			compatible = "fixed-partitions";
			#address-cells = <1>;
			#size-cells = <1>;

			partition@0 {
				label = "u-boot-bak";
				reg = <0x0 0x80000>;
				read-only;
			};

			partition@80000 {
				label = "u-boot";
				reg = <0x080000 0x80000>;
				read-only;
			};

			partition@100000 {
				label = "u-boot-env0";
				reg = <0x100000 0x40000>;
				read-only;
			};

			partition@140000 {
				label = "u-boot-env1";
				reg = <0x140000 0x40000>;
				read-only;
			};

			partition@180000 {
				label = "nvram4";
				reg = <0x180000 0x40000>;
			};

			partition@1c0000 {
				label = "nvram3";
				reg = <0x1c0000 0x40000>;
			};

			partition@200000 {
				label = "nvram2";
				reg = <0x200000 0x40000>;
			};

			partition@240000 {
				label = "nvram1";
				reg = <0x240000 0x40000>;
			}; 

			partition@280000 {
				label = "firmware";
				compatible = "denx,uimage";
				reg = <0x280000 0x3cc0000>;
			};			
		};
	};
};

Generic.mk

define Device/extreme_ap3805i
  SOC := qca9557
  DEVICE_VENDOR := Extreme Networks
  DEVICE_MODEL := AP3805i
  DEVICE_PACKAGES := kmod-ath10k-ct ath10k-firmware-qca988x-ct
  IMAGE_SIZE := 62208k
endef
TARGET_DEVICES += extreme_ap3805i

02_network

	extreme,ap3805i|\
	enterasys,ws-ap3705i)
		label_mac=$(mtd_get_mac_ascii u-boot-env0 ethaddr)
		;;

The device stays on now with the watchdog on GPIO16.

Now the focus is on the wireless drivers. I can't seem to get neither of them to work and from what i can get is because of the cal-stuff.

root@OpenWrt:/# dmesg | grep 'ath10k'
[   11.334741] ath10k 5.8 driver, optimized for CT firmware, probing pci device: 0x3c.
[   11.355652] ath10k_pci 0000:00:00.0: enabling device (0000 -> 0002)
[   11.364259] ath10k_pci 0000:00:00.0: pci irq legacy oper_irq_mode 1 irq_mode 0 reset_mode 0
[   14.870766] ath10k_pci 0000:00:00.0: qca988x hw2.0 target 0x4100016c chip_id 0x043222ff sub 0000:0000
[   14.880164] ath10k_pci 0000:00:00.0: kconfig debug 0 debugfs 1 tracing 0 dfs 1 testmode 0
[   14.892385] ath10k_pci 0000:00:00.0: firmware ver 10.1-ct-8x-__fW-022-ecad3248 api 2 features wmi-10.x,has-wmi-mgmt-tx,mfp,txstatus-noack,wmi-10.x-CT,ratemask-CT,txrate-CT,get-temp-CT,tx-rc-CT,cust-stats-CT,retry-gt2-CT,txrate2-Cf
[   15.224795] ath10k_pci 0000:00:00.0: board_file api 1 bmi_id N/A crc32 bebc7c08
[   15.284538] ath10k_pci 0000:00:00.0: otp calibration failed: 2
[   15.290482] ath10k_pci 0000:00:00.0: failed to run otp: -22 (download-cal-data)
[   15.297909] ath10k_pci 0000:00:00.0: could not init core (-22)
[   15.322944] ath10k_pci 0000:00:00.0: could not probe fw (-22)

The device don't have any ART/calibration-partition and when doing hexdumps of the mtd-partitions there seem to be none there either.
The OEM dmesg show the following:

 scn->cal_mem a3fc4000 AH_CAL_LOCATIONS_PCI 3fc4000 HOST_CALDATA_SIZE 4000
ol_target_init() BMI inited.
ol_target_init() BMI Get Target Info.
Chip id: 0x7, chip version: 0x4100016c

 CE WAR Disabled
NUM_DEV=1 FWMODE=0x2 FWSUBMODE=0x0 FWBR_BUF 0
ol_target_init() configure Target .
ol_transfer_bin_file: flash data file defined

 NOR FLASH Wifi0 Select OFFSET 1000
qc98xx_verify_checksum: flash checksum passed: 0xc43f
ol_transfer_bin_file 4031: Download Flash data len 2116
Board extended Data download address: 0x0

 Mission mode: Firmware CHIP Version 0
ol_transfer_bin_file: Downloading firmware file: AR9888/hw.2/athwlan.bin
ol_transfer_bin_file 3851: downloading file 1, Download data len 229368
ol_target_init() Download FW done.

And with some testing i found that after a full OEM OS boot i can reboot into uboot and find the cal-data at
'0xa3fc4000' + 1000 offset.
But if i pull the plug and cold boot into uboot, it's not there.
Any idea to what that register points? Is it now showing up after a cold-boot because of the pci-init?

mpratt14 · January 3, 2021, 11:27pm

Not sure right now...but let's start with completing the DTS with what we know...

also here is linux documentation for ath10k in DTS
there are some properties for manually setting calibration data, which might be what you need
https://www.kernel.org/doc/Documentation/devicetree/bindings/net/wireless/qcom,ath10k.txt

&eth0 {
	status = "okay";

	phy-handle = <&phy5>;
	phy-mode = "rgmii-id";

	pll-data = <0x82000000 0x80000101 0x80001313>;

	gmac-config {
		device = <&gmac>;
		rgmii-enabled = <1>;
	};
};

&pcie0 {
	status = "okay";

	wifi@0,0,0 {
		compatible = "qcom,ath10k";
		reg = <0x0 0 0 0 0>;
	};
};

Dragets · January 4, 2021, 10:20pm

That's done now but same cal-error. Neither of Ath9k and ath10k is starting. Eth is coming up at 1000 full duplex so thats ok for now until further testing is done.

As far as the 9882-card itself goes it's definitely a Senao-made one.

What I get from the link you provided is that the cal-properties in DT is only for pushing them in that way?

mpratt14 · January 5, 2021, 12:00am

I am not totally sure...but it looks like those properties are for providing the cal-data

so if you can somehow get your hands on it, you can put it in the DTS in hex

also, is it somehow possible that there's another flash chip on the board?

Dragets · January 5, 2021, 8:02pm

Ok, so i'm starting to figure this out. The calibration-data is actually on the flash chip.

bdinfo gave me:

flashstart  = 0x9F000000
flashsize   = 0x04000000

And the OEM-dmesg put out:

scn->cal_mem a3fc4000 AH_CAL_LOCATIONS_PCI 3fc4000 HOST_CALDATA_SIZE 4000

At the same time, the OEM-OS mounts this:

Creating 11 MTD partitions on "ath-nor0":
0x000000000000-0x000000080000 : "BootBAK"
0x000000080000-0x000000100000 : "BootPRI"
0x000000100000-0x000000140000 : "CFG1"
0x000000140000-0x000000180000 : "CFG2"
0x000000180000-0x0000001c0000 : "NVRAM4"
0x0000001c0000-0x000000200000 : "NVRAM3"
0x000000200000-0x000000240000 : "NVRAM2"
0x000000240000-0x000000280000 : "NVRAM1"
0x000000280000-0x000001280000 : "PriImg"
0x000001280000-0x000002280000 : "SecImg"
0x000002280000-0x000003f40000 : "FS"

As you can see, there are space left in the end.
9f000000+3f40000=a2f40000
9f000000+4000000=a3000000
And here it comes:

9f000000+3fc4000 (From dmesg) = a2fc4000 = is within the flashsize but not in any MTD-parts.

If i cold start the unit and just enter U-Boot directly i can find the cal-data for ATH9K at a2fc1000 and for ATH10K at a2fc5000.
And there it is, the magic numbers 0x1000 and 0x5000.
Now i just need to create the partition in the DTS and then fix the caldata-files hopefully.

Edit:
And as i thought, ATH10K working. Now on to wmac.

root@OpenWrt:/# dmesg | grep 'ath10k'
[   11.200025] ath10k 5.8 driver, optimized for CT firmware, probing pci device: 0x3c.
[   11.219719] ath10k_pci 0000:00:00.0: enabling device (0000 -> 0002)
[   11.228372] ath10k_pci 0000:00:00.0: pci irq legacy oper_irq_mode 1 irq_mode 0 reset_mode 0
[   14.817627] ath10k_pci 0000:00:00.0: qca988x hw2.0 target 0x4100016c chip_id 0x043222ff sub 0000:0000
[   14.827032] ath10k_pci 0000:00:00.0: kconfig debug 0 debugfs 1 tracing 0 dfs 1 testmode 0
[   14.839259] ath10k_pci 0000:00:00.0: firmware ver 10.1-ct-8x-__fW-022-ecad3248 api 2 features wmi-10.x,has-wmi-mgmt-tx,mfp,txstatus-noack,wmi-10.x-CT,ratemask-CT,txrate-CT,get-temp-CT,tx-rc-CT,cust-stats-CT,retry-gt2-CT,txrate2-Cf
[   15.172552] ath10k_pci 0000:00:00.0: board_file api 1 bmi_id N/A crc32 bebc7c08
[   16.089916] ath10k_pci 0000:00:00.0: 10.1 wmi init: vdevs: 16  peers: 127  tid: 256
[   16.107706] ath10k_pci 0000:00:00.0: wmi print 'P 128 V 8 T 410'
[   16.114039] ath10k_pci 0000:00:00.0: wmi print 'msdu-desc: 1424  sw-crypt: 0 ct-sta: 0'
[   16.122192] ath10k_pci 0000:00:00.0: wmi print 'alloc rem: 24984 iram: 38672'
[   16.194041] ath10k_pci 0000:00:00.0: htt-ver 2.1 wmi-op 2 htt-op 2 cal file max-sta 128 raw 0 hwcrypto 1
[   16.203994] ath10k_pci 0000:00:00.0: NOTE:  Firmware DBGLOG output disabled in debug_mask: 0x10000000

This added to the dts:

			partition@3fc0000 {
				label = "art";
				reg = <0x3fc0000 0x40000>;
				read-only;

mpratt14 · January 5, 2021, 9:37pm

Good job, I was about to say that chip you pictured looks like a linear regulator or step-down power supply or some other power related chip, because it is next to 2 large capacitors and an inductor

Since you had a hard time finding the cal data, it may be possible that the wireless devices start up with the wrong MAC address, so you can use the DTS to give it the right address probably, otherwise using the functions available in these scripts
target/linux/ath79/generic/base-files/etc/hotplug.d/firmware/10-ath9k-eeprom
target/linux/ath79/generic/base-files/etc/hotplug.d/firmware/11-ath10k-caldata

Dragets · January 5, 2021, 9:56pm

Thank you.
Yeah i was implementing them and now both wifi's are working with good performance.

5ghz: Speedtest giving about same as my wired, docked laptop. ~300Mbit up and down. 1ms ping and 1ms jitter, 0,0% loss.
2.4ghz: 42down and 66up, 1ms ping, 1ms jitter, 0,0% loss.

My hopefully last question before doing a pull request on Github.

After adding the GPIO16-watchdog i'm getting this error on boot:

[    5.359347] ath79-wdt 18060008.wdt: unable to register misc device, err=-16
[    5.366459] ath79-wdt: probe of 18060008.wdt failed with error -16
...
[    5.998146] init: - watchdog -

Is that something to worry about? Should i disable ath79-wdt fully in the DT?

Current bootlog:

Boot (PRI)-> run ramboot_openwrt
dup 1 speed 1000
*** Warning: no boot file name; using '1D01A8C0.img'
Using eth0 device
TFTP from server 192.168.1.66; our IP address is 192.168.1.29
Filename '1D01A8C0.img'.
Load address: 0x89000000
Loading: #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         ###
done
Bytes transferred = 5668543 (567ebf hex)
## Booting image at 89000000 ...
   Image Name:   MIPS OpenWrt Linux-5.4.86
   Created:      2021-01-02  13:36:25 UTC
   Image Type:   MIPS Linux Kernel Image (lzma compressed)
   Data Size:    5668479 Bytes =  5.4 MB
   Load Address: 80060000
   Entry Point:  80060000
   Verifying Checksum ... OK
   Uncompressing Kernel Image ... OK

Starting kernel ...

[    0.000000] Linux version 5.4.86 (ahe@aheserver) (gcc version 8.4.0 (OpenWrt GCC 8.4.0 r15404-55e23f2c02)) #0 Sat Jan 2 13:36:25 2021
[    0.000000] printk: bootconsole [early0] enabled
[    0.000000] CPU0 revision is: 00019750 (MIPS 74Kc)
[    0.000000] MIPS: machine is Extreme Networks AP3805i
[    0.000000] SoC: Qualcomm Atheros QCA9558 ver 1 rev 0
[    0.000000] Initrd not found or empty - disabling initrd
[    0.000000] Primary instruction cache 64kB, VIPT, 4-way, linesize 32 bytes.
[    0.000000] Primary data cache 32kB, 4-way, VIPT, cache aliases, linesize 32 bytes
[    0.000000] Zone ranges:
[    0.000000]   Normal   [mem 0x0000000000000000-0x000000000fffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000000000-0x000000000fffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x000000000fffffff]
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 64960
[    0.000000] Kernel command line: console=ttyS0,115200n8 rootfstype=squashfs,jffs2
[    0.000000] Dentry cache hash table entries: 32768 (order: 5, 131072 bytes, linear)
[    0.000000] Inode-cache hash table entries: 16384 (order: 4, 65536 bytes, linear)
[    0.000000] Writing ErrCtl register=00000000
[    0.000000] Readback ErrCtl register=00000000
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] Memory: 248432K/262144K available (4893K kernel code, 191K rwdata, 1124K rodata, 4728K init, 196K bss, 13712K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[    0.000000] NR_IRQS: 51
[    0.000000] random: get_random_bytes called from start_kernel+0x32c/0x520 with crng_init=0
[    0.000000] CPU clock: 720.000 MHz
[    0.000000] clocksource: MIPS: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 5309056796 ns
[    0.000007] sched_clock: 32 bits at 360MHz, resolution 2ns, wraps every 5965232126ns
[    0.008294] Calibrating delay loop... 359.42 BogoMIPS (lpj=718848)
[    0.042886] pid_max: default: 32768 minimum: 301
[    0.047930] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.055717] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.067952] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.078364] futex hash table entries: 256 (order: -1, 3072 bytes, linear)
[    0.085711] pinctrl core: initialized pinctrl subsystem
[    0.092112] NET: Registered protocol family 16
[    0.121803] workqueue: max_active 576 requested for napi_workq is out of range, clamping between 1 and 512
[    0.134974] clocksource: Switched to clocksource MIPS
[    0.141410] NET: Registered protocol family 2
[    0.146719] tcp_listen_portaddr_hash hash table entries: 512 (order: 0, 4096 bytes, linear)
[    0.155676] TCP established hash table entries: 2048 (order: 1, 8192 bytes, linear)
[    0.163861] TCP bind hash table entries: 2048 (order: 1, 8192 bytes, linear)
[    0.171387] TCP: Hash tables configured (established 2048 bind 2048)
[    0.178268] UDP hash table entries: 256 (order: 0, 4096 bytes, linear)
[    0.185269] UDP-Lite hash table entries: 256 (order: 0, 4096 bytes, linear)
[    0.192965] NET: Registered protocol family 1
[    0.197655] PCI: CLS 0 bytes, default 32
[    1.082981] random: fast init done
[    4.806317] workingset: timestamp_bits=14 max_order=16 bucket_order=2
[    4.819090] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    4.825316] jffs2: version 2.2 (NAND) (SUMMARY) (LZMA) (RTIME) (CMODE_PRIORITY) (c) 2001-2006 Red Hat, Inc.
[    4.849654] pinctrl-single 1804002c.pinmux: 544 pins, size 68
[    4.856719] Serial: 8250/16550 driver, 1 ports, IRQ sharing disabled
[    4.864130] printk: console [ttyS0] disabled
[    4.868751] 18020000.uart: ttyS0 at MMIO 0x18020000 (irq = 9, base_baud = 2500000) is a 16550A
[    4.877941] printk: console [ttyS0] enabled
[    4.877941] printk: console [ttyS0] enabled
[    4.886838] printk: bootconsole [early0] disabled
[    4.886838] printk: bootconsole [early0] disabled
[    4.902893] spi-nor spi0.0: s25fl512s (65536 Kbytes)
[    4.908031] 10 fixed-partitions partitions found on MTD device spi0.0
[    4.914569] Creating 10 MTD partitions on "spi0.0":
[    4.919527] 0x000000000000-0x000000080000 : "u-boot-bak"
[    4.925708] 0x000000080000-0x000000100000 : "u-boot"
[    4.931594] 0x000000100000-0x000000140000 : "cfg1"
[    4.937273] 0x000000140000-0x000000180000 : "cfg2"
[    4.942953] 0x000000180000-0x0000001c0000 : "nvram4"
[    4.948811] 0x0000001c0000-0x000000200000 : "nvram3"
[    4.954680] 0x000000200000-0x000000240000 : "nvram2"
[    4.960534] 0x000000240000-0x000000280000 : "nvram1"
[    4.966383] 0x000000280000-0x000003f40000 : "firmware"
[    4.981629] 0x000003fc0000-0x000004000000 : "art"
[    4.988915] libphy: Fixed MDIO Bus: probed
[    4.995894] ag71xx 19000000.eth: invalid MAC address, using random address
[    5.316765] libphy: ag71xx_mdio: probed
[    5.338422] ag71xx 19000000.eth: connected to PHY at mdio.0:05 [uid=004dd072, driver=Atheros 8035 ethernet]
[    5.348876] eth0: Atheros AG71xx at 0xb9000000, irq 4, mode: rgmii-id
[    5.355624] i2c /dev entries driver
[    5.359347] ath79-wdt 18060008.wdt: unable to register misc device, err=-16
[    5.366459] ath79-wdt: probe of 18060008.wdt failed with error -16
[    5.374324] NET: Registered protocol family 10
[    5.382912] Segment Routing with IPv6
[    5.386779] NET: Registered protocol family 17
[    5.391373] 8021q: 802.1Q VLAN Support v1.8
[    5.396401] PCI host bridge /ahb/pcie-controller@180c0000 ranges:
[    5.402645]  MEM 0x0000000010000000..0x0000000011ffffff
[    5.407963]   IO 0x0000000000000000..0x0000000000000000
[    5.413423] PCI host bridge to bus 0000:00
[    5.417620] pci_bus 0000:00: root bus resource [mem 0x10000000-0x11ffffff]
[    5.424601] pci_bus 0000:00: root bus resource [io  0x0000]
[    5.430257] pci_bus 0000:00: root bus resource [??? 0x00000000 flags 0x0]
[    5.437147] pci_bus 0000:00: No busn resource found for root bus, will use [bus 00-ff]
[    5.445218] pci 0000:00:00.0: [168c:003c] type 00 class 0x028000
[    5.451366] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x001fffff 64bit]
[    5.458308] pci 0000:00:00.0: reg 0x30: [mem 0x00000000-0x0000ffff pref]
[    5.465186] pci 0000:00:00.0: supports D1
[    5.469261] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[    5.476094] pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to 00
[    5.482851] pci 0000:00:00.0: BAR 0: assigned [mem 0x10000000-0x101fffff 64bit]
[    5.490296] pci 0000:00:00.0: BAR 6: assigned [mem 0x10200000-0x1020ffff pref]
[    5.515749] Freeing unused kernel memory: 4728K
[    5.520372] This architecture does not have kernel memory protection.
[    5.526902] Run /init as init process
[    5.994403] init: Console is alive
[    5.998146] init: - watchdog -
[    6.018312] kmodloader: loading kernel modules from /etc/modules-boot.d/*
[    6.027884] kmodloader: done loading kernel modules from /etc/modules-boot.d/*
[    6.045761] init: - preinit -
[    6.428645] random: jshn: uninitialized urandom read (4 bytes read)
[    6.538780] random: jshn: uninitialized urandom read (4 bytes read)
[    6.723888] random: jshn: uninitialized urandom read (4 bytes read)
Press the [f] key and hit [enter] to enter failsafe mode
Press the [1], [2], [3] or [4] key and hit [enter] to select the debug level
[   10.984441] eth0: link up (1000Mbps/Full duplex)
[   10.991487] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[   11.029637] eth0: link down
[   11.042947] procd: - early -
[   11.046007] procd: - watchdog -
[   11.604819] procd: - watchdog -
[   11.608373] procd: - ubus -
[   11.618472] urandom_read: 5 callbacks suppressed
[   11.618479] random: ubusd: uninitialized urandom read (4 bytes read)
[   11.661793] random: ubusd: uninitialized urandom read (4 bytes read)
[   11.669516] procd: - init -
Please press Enter to activate this console.
[   12.220187] kmodloader: loading kernel modules from /etc/modules.d/*
[   12.258920] Loading modules backported from Linux version v5.8.18-0-gab435ce49bd1
[   12.266553] Backport generated by backports.git v5.8.18-1-0-ga630fd46
[   12.322500] xt_time: kernel timezone is -0000
[   12.447014] PPP generic driver version 2.4.2
[   12.456414] NET: Registered protocol family 24
[   12.465713] urngd: v1.0.2 started.
[   12.530851] ath10k 5.8 driver, optimized for CT firmware, probing pci device: 0x3c.
[   12.551746] ath10k_pci 0000:00:00.0: enabling device (0000 -> 0002)
[   12.560406] ath10k_pci 0000:00:00.0: pci irq legacy oper_irq_mode 1 irq_mode 0 reset_mode 0
[   12.691999] random: crng init done
[   16.134268] ath10k_pci 0000:00:00.0: qca988x hw2.0 target 0x4100016c chip_id 0x043222ff sub 0000:0000
[   16.143671] ath10k_pci 0000:00:00.0: kconfig debug 0 debugfs 1 tracing 0 dfs 1 testmode 0
[   16.155898] ath10k_pci 0000:00:00.0: firmware ver 10.1-ct-8x-__fW-022-ecad3248 api 2 features wmi-10.x,has-wmi-mgmt-tx,mfp,txstatus-noack,wmi-10.x-CT,ratemask-CT,txrate-CT,get-temp-CT,tx-rc-CT,cust-stats-CT,retry-gt2-CT,txrate2-Cf
[   16.489178] ath10k_pci 0000:00:00.0: board_file api 1 bmi_id N/A crc32 bebc7c08
[   17.418719] ath10k_pci 0000:00:00.0: 10.1 wmi init: vdevs: 16  peers: 127  tid: 256
[   17.436506] ath10k_pci 0000:00:00.0: wmi print 'P 128 V 8 T 410'
[   17.442841] ath10k_pci 0000:00:00.0: wmi print 'msdu-desc: 1424  sw-crypt: 0 ct-sta: 0'
[   17.450990] ath10k_pci 0000:00:00.0: wmi print 'alloc rem: 24984 iram: 38672'
[   17.522937] ath10k_pci 0000:00:00.0: htt-ver 2.1 wmi-op 2 htt-op 2 cal file max-sta 128 raw 0 hwcrypto 1
[   17.532891] ath10k_pci 0000:00:00.0: NOTE:  Firmware DBGLOG output disabled in debug_mask: 0x10000000
[   17.704257] ath9k 18100000.wmac: Direct firmware load for ath9k-eeprom-ahb-18100000.wmac.bin failed with error -2
[   17.714718] ath9k 18100000.wmac: Falling back to sysfs fallback for: ath9k-eeprom-ahb-18100000.wmac.bin
[   18.282590] ieee80211 phy1: Atheros AR9550 Rev:0 mem=0xb8100000, irq=13
[   18.317917] kmodloader: done loading kernel modules from /etc/modules.d/*
[   71.277312] br-lan: port 1(eth0) entered blocking state
[   71.282668] br-lan: port 1(eth0) entered disabled state
[   71.288253] device eth0 entered promiscuous mode
[   74.344341] eth0: link up (1000Mbps/Full duplex)
[   74.349205] br-lan: port 1(eth0) entered blocking state
[   74.354529] br-lan: port 1(eth0) entered forwarding state
[   74.361690] IPv6: ADDRCONF(NETDEV_CHANGE): br-lan: link becomes ready

mpratt14 · January 5, 2021, 10:09pm

I wouldn't worry about the internal watchdog error, but you can mention it in the pull request

you can now add 0x80000 in length to "firmware" so there's no wasted space

then you can finally backup all partitions in LuCI and start flashing

check MAC address for eth0 with ifconfig and compare to device label

you can also check wireless devices MAC that way too, but they must be enabled first

mpratt14 · January 5, 2021, 11:49pm

on second thought, you might want to make sure there is nothing in this space between firmware and your new art partition before you do this

and alternatively, you can make the ART partition start at 0x3f40000 so that it aligns better with how OEM partitions originally are