DeviceTree for OpenWRT on WatchGuard T35

For Developers
1

I am trying to add support for OpenWRT on a WatchGuard T35. The T35 uses a NXP T1024 SoC, which uses a e5500 CPU and has a PowerPC64 architecture.

After about a 100 hours of work - I have gotten OpenWRT to boot on a WatchGuard T35 - yippee!
I have have the flattened device tree .dts from the stock WatchGuard installation that I can turn into the device tree source .dts file, but of course the labels are gone.

The stock image use a v3.12 linux kernel. I'm working on getting OpenWRT v22.03.7 (linux kernel v5.10.221) working before working on getting the latest version of OpenWRT running. I belive the changes between linux v3.12 and 5.10.221 have changed sufficiently that I can't use the stock dtb anymore.

The device boots, and I get to userspace, but I have no Ethernet ports.
ip a shows only the loop-back interface.

Further, dmesg show the drivers failed to register for the Ethernet ports:

[    0.116718] OF: /soc@ffe000000/fman@400000: could not get #clock-cells for /cpus/PowerPC,e5500@0/l2-cache
[    0.126123] OF: /soc@ffe000000/fman@400000: could not get #clock-cells for /cpus/PowerPC,e5500@0/l2-cache
...
[    2.831597] fsl_dpaa_mac ffe4e4000.ethernet: dev_get_drvdata(/soc@ffe000000/fman@400000/port@89000) failed
[    2.841329] fsl_dpaa_mac: probe of ffe4e4000.ethernet failed with error -22
[    2.848397] fsl_dpaa_mac ffe4e6000.ethernet: dev_get_drvdata(/soc@ffe000000/fman@400000/port@8a000) failed
[    2.858103] fsl_dpaa_mac: probe of ffe4e6000.ethernet failed with error -22

How can I identify the correct device tree parameters?

I've made a few changes to the DTS to get this far, namely:
adding

      clocks = <&clockgen 3 0>;
      clock-names = "fmanclk";

to my fman@400000 node in the dts, with clockgen being a label for the global-utilies@e1000
as it is in linux/arch/powerpc/boot/dts/fsl/qoriq-clockgen{1,2}.dtsi

adding

      fsl,ucode-addr = <0x1110000 0x2000>;

also to the fman node - that is the address of the fman ucode parition

adding labels to the port fman-port-1g-rx/tx nodes under fman and then

 fman0_rx_0x0a: port@89000 {
	compatible = "fsl,fman-port-1g-tx";
 	...      
 }
			
fman0_tx_0x2a: port@a9000 {...}
fman1_rx_0x0a: port@8a000 {...}
fman1_tx_0x2a: port@aa000 {...}

so that I can use them in the fman/ethernet nodes as fman-ports:

ethernet@e4000 {
fsl,fman-ports = <&fman0_rx_0x0a &fman0_tx_0x2a>;
}
ethernet@e6000 {
fsl,fman-ports = <&fman1_rx_0x0a &fman1_tx_0x2a>;
}

Again, I modeled this after linux/arch/powerpc/boot/dts/fsl/qoriq-fman{,3}-0-1g-2.dtsi

I have the T1024 reference design board (t1024rdb) documentation from NXP, as well - though I cannot share it here.

2

Here's the entire DTS on pastebin with expiry 2026-11-03

3

I have gathered that the error code -22 is an EINVAL (bad value error) specifically from
this block at lines 766-773 of mac.c (https://github.com/torvalds/linux/blob/v5.10/drivers/net/ethernet/freescale/fman/mac.c#L766)

	mac_dev->port[i] = fman_port_bind(&of_dev->dev);
	if (!mac_dev->port[i]) {
		dev_err(dev, "dev_get_drvdata(%pOF) failed\n",
			dev_node);
		err = -EINVAL;
		goto _return_of_node_put;
	}
	of_node_put(dev_node);

Which means the fman_port_bind is returning an error value. It's defined in fman_port.c (https://github.com/torvalds/linux/blob/v5.10/drivers/net/ethernet/freescale/fman/fman_port.c#L1711)

struct fman_port *fman_port_bind(struct device *dev)
{
	return (struct fman_port *)(dev_get_drvdata(get_device(dev)));
}

which means dev_get_drvdata is returning an error value. dev_get_drvdata is part of the linux kernel in device.h not specific to the driver:
(https://github.com/torvalds/linux/blob/master/include/linux/device.h#L799)

static inline void *dev_get_drvdata(const struct device *dev)
{
	return dev->driver_data;
}

The descrpiton of device->driver_data from https://github.com/torvalds/linux/blob/master/include/linux/device.h#L582 is

	void		*driver_data;	/* Driver data, set and get with dev_set_drvdata/dev_get_drvdata */

So beyond that, I'm not sure where the field is being set. I'm having trouble finding the constructor for the device being passed to fman_port_bind.

Is this indicative of a missing driver for the device or a bad value being passed from the device tree?

4

you are going too deep.

struct device “is not not a place you should look for” . This is general for all device drivers. Call this “metadata” or “internal” or so.

dev_get_drvdata() and dev_set_drvdata() are the proper “functions” to “access” ->driver_data

in ->driver_data contains “extra” data specific for a device. i.e. mdio-bus

for compatible = "fsl,fman-memac";

look here { .compatible = "fsl,fman-memac", .data = memac_initialization },

int memac_initialization(struct mac_device *mac_dev,
			 struct device_node *mac_node,
			 struct fman_mac_params *params)

also

git grep "fsl,fman-memac" Documentation/

Documentation/devicetree/bindings/net/fsl,fman-dtsec.yaml
Documentation/devicetree/bindings/net/fsl,fman-mdio.yaml
5

Shameless plug: Once you get this going, since I assume the T35, like T10/T15/T20/T30/T40/T50 is also using a recovery firmware <=12.5.9U1/12.7.2U1 (ie. the versions before they introduced signed images): There's still the task of reverse-engineering the stock firmware format, in order to create an OpenWrt factory image recipe.

I have almost everything - what I'm missing is the SHA1-HMAC at the end. So far, I wasn't able to figure out the HMAC Secret nor which parts of the image are actually hashed to generate that SHA1.

Would be glad for any input.

6

Many thanks. What I found is that that the two issues I was having were, in fact, missing drivers.
I was baffled by how the T35, with 5 physical Ethernet ports, could function with only one phy handle in the device tree. It is because unlike the T1024rdb, the T35 use a Marvel 88E6172 on-board switch to connect the 5 Ethernet ports to the SoC. Indeed, the SoC has only one physical handle.
The original DTS has

/ {
  soc@ffe00000 {
    fman@40000 {
      ethernet@e0000 {
         status = "disabled"
      };  
      ethernet@e2000 {
         status = "disabled"
      };  
      fm1mac3: ethernet@e4000 {
         phy-connection-type = "rgmii"
      };  
      fm1mac4: ethernet@e6000 {
         phy-connection-type = "rgmii"
      };  
      mdio@fd000 {
        ethernet-phy@1 {
          compatible = "ethernet-phy-ieee802.3-c45";
        };         
      };  
    };
  };
  &enet2 {
    fsl,fman-mac = <&fm1mac3>;
  };
  &enet3 {
    fsl,fman-mac = <&fm1mac4>;
  };
};

And is otherwise almost the same as the t1024rdb.dts that's a part of the public linux kernel

The two drivers I had accidentally disabled were enabled by linux kernel configs:
CONFIG_NET_DSA_MV88E6XXX=y
CONFIG_CLK_QORIQ=y

I'm still troubleshooting why I don't see the 5 ports connected to the swtich, I only had the revelation to enable the MV88E6XXX driver this morning

7

Could you elaborate on what specifically you mean by the firmware? I'll share what I've been doing and discover thus far.

The Uboot menu has 3 options:
WatchGuard (SYSA)
WatchGuard (SYSB Recovery/Diagnostic Mode)
WatchGuard (SYSA from /boot)

The stock T35 SSD has 7 partitions

  1. [0x100000, 0x1100000) This is the /boot partition, as far as I can tell. You boot the kernel and DTS image here by selecting the "SYSA from /boot" option
    It has an empty file FIRSTBOOT, a device tree t35.dts, two copies of the uImage kernel uImage and uImage_t35 but uImage is the one that's used by Uboot by default. Formated as EXT2

  2. [0x1100000, 0x9100000) This is the SYSB partition used for recovery and is accessed using the "SYSB Recovery/Diagnostic Mode" Uboot menu option
    Also EXT2, similar to boot with files:

dev       initrd      sbin          t35.dtb  uImage_t35
info.txt  lost+found  sysb_root.gz  uImage
  1. [0x9100000, 0x55200000) This is the SYSA partiton that is booted into by default,
    The way I've been booting OpenWRT is by overwriting this partition: putting my OpenWRT root file system here with a DTB and kernel named exactly as in the stock image.
    That seems to circumvent the filesystem check, as I believe that is run long after UBoot passes off to the kernel.
    It's also EXT2 and has the runtime root filesystem with files:
bin    dev   info.txt  lib64       pending  sbin         sysfs    tools       usr
boot   etc   initrd    lost+found  proc     softparse.h  t35.dtb  uImage      var
data2  home  lib       opt         root     sys          tmp      uImage_t35
  1. The 4th, partition is empty, EXT3, and I don't know what it does
  2. The rest of the partions I haven't really paid attention to

If you instead meant the m25p80 flash, that also has some partions starting at base address 0xffe110000:

				partition@00000000 {
					reg = <0x00 0x100000>;
					label = "NOR (RW) WG U-Boot Image";
				};

				partition@00100000 {
					reg = <0x100000 0x10000>;
					label = "NOR (RW) WG U-Boot ENV";
				};

				partition@00110000 {
					reg = <0x110000 0x20000>;
					label = "NOR (RW) FMAN UCODE";
				};

				partition@00130000 {
					reg = <0x130000 0xd0000>;
					label = "NOR (RW) WG EMPYT-HOLE";
				};

				partition@00200000 {
					reg = <0x200000 0x20000>;
					label = "NOR (RW) WG CFG0";
				};

				partition@00220000 {
					reg = <0x220000 0x10000>;
					label = "NOR (RW) WG CFG1";
				};

				partition@00230000 {
					reg = <0x230000 0x20000>;
					label = "NOR (RW) WG MFG";
				};

				partition@00250000 {
					reg = <0x250000 0xb0000>;
					label = "NOR (RW) WG bootOpt Data & reserved";
				};

				partition@00300000 {
					reg = <0x300000 0x100000>;
					label = "NOR (RW) WG extra reserved";
				};
8

Also of note, the stock UBoot bootloader is on the first partition of the m25p80 NOR flash. The Uboot command line is password protected. The entire bootenv args are here:

But TL;DR; it uses
fsaddr=4000000
loadaddr=1000000
bootm $loadaddr - $fdtaddr;

I also miswrote earlier, the kernel file used by UBoot is the uImage_t35, not uImage

9

I was talking about the stock firmware file format (the *.sysa-dl file from the vendor), not how the flash is organized and not whether Uboot is passworded.

10

There are no files matching *.sysa-dl in the image I have. I guess the T35 doesn't implement that feature. I misunderstood,

11

Again git tells you more ..

git grep NET_DSA_MV88E6XXX

drivers/net/dsa/mv88e6xxx/port.h:584:#ifdef CONFIG_NET_DSA_MV88E6XXX_LEDS
drivers/net/dsa/mv88e6xxx/ptp.h:67:#ifdef CONFIG_NET_DSA_MV88E6XXX_PTP
drivers/net/dsa/mv88e6xxx/ptp.h:79:#else /* !CONFIG_NET_DSA_MV88E6XXX_PTP */
drivers/net/dsa/mv88e6xxx/ptp.h:94:#endif /* CONFIG_NET_DSA_MV88E6XXX_PTP */

with this you will find the probe function and the related “device entry” here i.e. "marvell,mv88e6085"

with this feed agian into git

git grep "marvell,mv88e6085"

now you have the related dts (for some other devices) your can copy from.

12

The firmware file that you download from the vendor is named *.sysa-dl. It is not a file that is stored on your device.

13

Indeed, I've been using the watchguard firebox m300 device config for comparison as it also uses a 5 ports connected to a Marvel mv88e6058 switch connected to the MDIO bus taht I believe is very simiar to the physical configuration of the T35. Relevant snippet here:

&mdio0 {
...
	switch0: switch@10 {
		compatible = "marvell,mv88e6085";
		reg = <0x10>;

		mdio {
			#address-cells = <1>;
			#size-cells = <0>;

			switch0phy0: switch0phy0@0 {
				reg = <0x00>;
				interrupt-parent = <&switch0>;
			};
...
			switch0phy4: switch0phy4@4 {
				reg = <0x04>;
				interrupt-parent = <&switch0>;
			};
		};

		ports {
			#address-cells = <1>;
			#size-cells = <0>;

			port@0 {
				reg = <0>;
				label = "sweth3";
				phy-handle = <&switch0phy0>;
			};
...
			port@4 {
				reg = <4>;
				label = "sweth7";
				phy-handle = <&switch0phy4>;
			};
...

//phy0-3 on the m30 don't exist on the T35. Nothing is connected directly to the MDIO bus except the switch

I have in my stock image DTS that the switch is connected at slot 0 on the MDIO bus instead of 10

14

sysfs will tell you more about your running system …

ls /sys/

or

ls /sys/bus

be warned this is quite exhaustive and the struture is recursive too ! Also some drivers/devices may appear without a “user”

Look for yourself