Debugging network interfaces on MT7621

glebzlat · November 4, 2024, 12:18pm

My device has 4 LAN (internal switch) and 2 WLAN (MT7603 and MT7662 on PCI) ports. I'm writing the firmware from the ground up. The problem is: interfaces seem to be created, but they are down.

ifconfig output:

ifconfig

br-lan    Link encap:Ethernet  HWaddr F8:5E:3C:1E:57:00  
          inet addr:192.168.1.1  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fd9a:bc00:6de7::1/60 Scope:Global
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

eth0      Link encap:Ethernet  HWaddr F8:5E:3C:1E:57:00  
          inet6 addr: fe80::fa5e:3cff:fe1e:5700/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1504  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:23 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:2670 (2.6 KiB)
          Interrupt:19 

lan1      Link encap:Ethernet  HWaddr F8:5E:3C:1E:57:00  
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

lan2      Link encap:Ethernet  HWaddr F8:5E:3C:1E:57:00  
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

lan3      Link encap:Ethernet  HWaddr F8:5E:3C:1E:57:00  
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:448 errors:0 dropped:0 overruns:0 frame:0
          TX packets:448 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:34944 (34.1 KiB)  TX bytes:34944 (34.1 KiB)

wan       Link encap:Ethernet  HWaddr F8:5E:3C:1E:57:00  
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Interrupt:19

ip link list output:

interfaces

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1504 qdisc mq state UP qlen 1000
    link/ether f8:5e:3c:1e:57:00 brd ff:ff:ff:ff:ff:ff
3: wan: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN qlen 1000
    link/ether f8:5e:3c:1e:57:00 brd ff:ff:ff:ff:ff:ff
4: lan0@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether f8:5e:3c:1e:57:00 brd ff:ff:ff:ff:ff:ff
5: lan1@eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master br-lan state LOWERLAYERDOWN qlen 1000
    link/ether f8:5e:3c:1e:57:00 brd ff:ff:ff:ff:ff:ff
6: lan2@eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master br-lan state LOWERLAYERDOWN qlen 1000
    link/ether f8:5e:3c:1e:57:00 brd ff:ff:ff:ff:ff:ff
7: lan3@eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master br-lan state LOWERLAYERDOWN qlen 1000
    link/ether f8:5e:3c:1e:57:00 brd ff:ff:ff:ff:ff:ff
11: br-lan: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN qlen 1000
    link/ether f8:5e:3c:1e:57:00 brd ff:ff:ff:ff:ff:ff
12: phy0-ap0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether f8:5e:3c:1e:57:00 brd ff:ff:ff:ff:ff:ff
13: phy1-ap0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
    link/ether 5c:3c:1e:57:01:ff brd ff:ff:ff:ff:ff:ff

dmesg output when I run /etc/init.d/network restart (the setup part):

dmesg

[  779.973012] mtk_soc_eth 1e100000.ethernet eth0: configuring for fixed/rgmii link mode
[  779.981375] mtk_soc_eth 1e100000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
[  779.988570] mt7530-mdio mdio-bus:1f lan1: configuring for phy/gmii link mode
[  779.998652] br-lan: port 1(lan1) entered blocking state
[  780.004144] br-lan: port 1(lan1) entered disabled state
[  780.009565] mt7530-mdio mdio-bus:1f lan1: entered allmulticast mode
[  780.016052] mtk_soc_eth 1e100000.ethernet eth0: entered allmulticast mode
[  780.025876] mt7530-mdio mdio-bus:1f lan1: entered promiscuous mode
[  780.052476] mt7530-mdio mdio-bus:1f lan2: configuring for phy/gmii link mode
[  780.060596] br-lan: port 2(lan2) entered blocking state
[  780.065957] br-lan: port 2(lan2) entered disabled state
[  780.071264] mt7530-mdio mdio-bus:1f lan2: entered allmulticast mode
[  780.080010] mt7530-mdio mdio-bus:1f lan2: entered promiscuous mode
[  780.096957] mt7530-mdio mdio-bus:1f lan3: configuring for phy/gmii link mode
[  780.105693] br-lan: port 3(lan3) entered blocking state
[  780.110962] br-lan: port 3(lan3) entered disabled state
[  780.116376] mt7530-mdio mdio-bus:1f lan3: entered allmulticast mode
[  780.124790] mt7530-mdio mdio-bus:1f lan3: entered promiscuous mode
[  780.275240] mtk_soc_eth 1e100000.ethernet wan: PHY [mt7530-0:04] driver [MediaTek MT7530 PHY] (irq=POLL)
[  780.284937] mtk_soc_eth 1e100000.ethernet wan: configuring for phy/rgmii link mode

Similar output when I try to turn on the WLAN (ip link set phy0-ap0 up):

dmesg

[ 1779.850167] br-lan: port 4(phy0-ap0) entered blocking state
[ 1779.855829] br-lan: port 4(phy0-ap0) entered disabled state
[ 1779.861457] mt7603e 0000:01:00.0 phy0-ap0: entered allmulticast mode
[ 1779.868342] mt7603e 0000:01:00.0 phy0-ap0: entered promiscuous mode

Other info:

contents of `/etc/board.json`

{
	"model": {
		"id": "cuman,spectra-ac12-g-v16",
		"name": "CUMAN SPECTRA V16"
	},
	"network": {
		"lan": {
			"ports": [
				"lan1",
				"lan2",
				"lan3",
				"lan4"
			],
			"protocol": "static"
		},
		"wan": {
			"device": "wan",
			"protocol": "dhcp"
		}
	},
	"system": {
		"compat_version": "1.1"
	},
	"wlan": {
		"phy0": {
			"path": "1e140000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0",
			"info": {
				"antenna_rx": 3,
				"antenna_tx": 3,
				"bands": {
					"2G": {
						"ht": true,
						"max_width": 40,
						"modes": [
							"NOHT",
							"HT20",
							"HT40"
						],
						"default_channel": 1
					}
				}
			}
		},
		"phy1": {
			"path": "1e140000.pcie/pci0000:00/0000:00:01.0/0000:02:00.0",
			"info": {
				"antenna_rx": 3,
				"antenna_tx": 3,
				"bands": {
					"5G": {
						"ht": true,
						"vht": true,
						"max_width": 80,
						"modes": [
							"NOHT",
							"HT20",
							"VHT20",
							"HT40",
							"VHT40",
							"VHT80"
						],
						"default_channel": 36
					}
				}
			}
		}
	}
}

device tree

#include "mt7621.dtsi"

#include <dt-bindings/gpio/gpio.h>
#include <dt-bindings/input/input.h>
#include <dt-bindings/leds/common.h>

/ {
	compatible = "device-name", "mediatek,mt7621-soc";
	model = "device model";

	aliases {
		label-mac-device = &gmac1;
		led-boot = &led_power_b;
		led-failsafe = &led_power_r;
		led-running = &led_power_g;
		led-upgrade = &led_power_g;
	};

	chosen {
		bootargs = "console=ttyS0,115200";
	};

	keys {
		compatible = "gpio-keys-polled";
		poll-interval = <20>;

		wps {
			label = "wps";
			gpios = <&gpio 18 GPIO_ACTIVE_LOW>;
			linux,code = <KEY_WPS_BUTTON>;
		};
	};

	leds {
		compatible = "gpio-leds";
		pinctrl-names = "default";

		led_power_r: power-r {
			label = "red:power-r";
			function = LED_FUNCTION_INDICATOR;
			color = <LED_COLOR_ID_RED>;
			gpios = <&gpio 26 GPIO_ACTIVE_LOW>;
		};

		led_power_g: power-g {
			label = "green:power-g";
			function = LED_FUNCTION_POWER;
			color = <LED_COLOR_ID_GREEN>;
			gpios = <&gpio 31 GPIO_ACTIVE_LOW>;
		};

		led_power_b: power-b {
			label = "blue:power-b";
			function = LED_FUNCTION_STATUS;
			color = <LED_COLOR_ID_BLUE>;
			gpios = <&gpio 27 GPIO_ACTIVE_LOW>;
		};

		wan {
			// Port 3
			label = "green:wan";
			function = LED_FUNCTION_WAN;
			color = <LED_COLOR_ID_GREEN>;
			gpios = <&gpio 14 GPIO_ACTIVE_LOW>;
		};

		lan1 {
			// Port 0
			label = "green:lan1";
			function = LED_FUNCTION_LAN;
			color = <LED_COLOR_ID_GREEN>;
			gpios = <&gpio 16 GPIO_ACTIVE_LOW>;
		};

		lan2 {
			// Port 1
			label = "green:lan2";
			function = LED_FUNCTION_LAN;
			color = <LED_COLOR_ID_GREEN>;
			gpios = <&gpio 15 GPIO_ACTIVE_LOW>;
		};

		lan3 {
			// Port 2
			label = "green:lan3";
			function = LED_FUNCTION_LAN;
			color = <LED_COLOR_ID_GREEN>;
			gpios = <&gpio 13 GPIO_ACTIVE_LOW>;
		};

		lan4 {
			// Port 2
			label = "green:lan4";
			function = LED_FUNCTION_LAN;
			color = <LED_COLOR_ID_GREEN>;
			gpios = <&gpio 17 GPIO_ACTIVE_LOW>;
		};

		wlan0 {
			// WiFi 2.4G Activity
			label = "green:wlan0";
			function = LED_FUNCTION_WLAN_2GHZ;
			color = <LED_COLOR_ID_GREEN>;
			gpios = <&gpio 33 GPIO_ACTIVE_LOW>;
		};

		wlan1 {
			// WiFi 5G Activity
			label = "green:wlan1";
			function = LED_FUNCTION_WLAN_5GHZ;
			color = <LED_COLOR_ID_GREEN>;
			gpios = <&gpio 28 GPIO_ACTIVE_LOW>;
		};
	};

	gpio_export {
		compatible = "gpio-export";
		#size-cells = <0>;

		usb-pwr {
			// USB pwr ctrl 1A
			gpio-export,name = "usb-pwr";
			gpio-export,output = <1>;
			gpios = <&gpio 22 GPIO_ACTIVE_HIGH>;
		};
	};

	i2c_pins {
		ralink,group = "i2c";
		ralink,function = "gpio";
	};

	i2c_ctrl {
		compatible = "i2c-gpio";
		status = "okay";
		// 3 SDA, 4 SCL
		gpios = <&gpio 3 (GPIO_ACTIVE_HIGH|GPIO_OPEN_DRAIN)
				 &gpio 4 (GPIO_ACTIVE_HIGH|GPIO_OPEN_DRAIN)>;
		i2c-gpio,delay-us = <20>;  // ~10 kHz
		#address-cells = <1>;
		#size-cells = <0>;

		atsha204@64 {
			compatible = "atmel,atsha204a";
			reg = <0x64>;
		};
	};
};

&palmbus {
	i2c@900 {
		status = "okay";
	};
};

&gpio {
	status = "okay";
};

&sdhci {
	status = "disabled";
};

&spi0 {
	status = "okay";

	flash@0 {
		compatible = "jedec,spi-nor";
		reg = <0>;
		spi-max-frequency = <10000000>;

		partitions {
			compatible = "fixed-partitions";
			#address-cells = <1>;
			#size-cells = <1>;

			partition@0 {
				label = "u-boot";
				reg = <0x0 0x30000>;
				read-only;
			};

			partition@30000 {
				label = "u-boot-env";
				reg = <0x30000 0x10000>;
				read-only;
			};

			factory: partition@40000 {
				label = "factory";
				reg = <0x40000 0x10000>;
				read-only;

				nvmem-layout {
					compatible = "fixed-layout";
					#address-cells = <1>;
					#size-cells = <1>;

					eeprom_factory_0000: eeprom@0000 {
						reg = <0x0000 0x8000>;
					};

					eeprom_factory_8000: eeprom@8000 {
						reg = <0x8000 0x8000>;
					};

					macaddr_factory_e000: mac-address@e000 {
						reg = <0xe000 0x6>;
					};

					macaddr_factory_e007: mac-address@e007 {
						reg = <0xe007 0x6>;
					};
				};
			};

			firmware: partition@50000 {
				compatible = "denx,uimage";
				label = "firmware";
				reg = <0x50000 0xfb0000>;
			};
		};
	};
};

&pcie {
	status = "okay";
};

&pcie0 {
	wifi0: wifi@0,0 {
		compatible = "pci14c3,7603";
		reg = <0x0000 0 0 0 0>;
		nvmem-cells = <&eeprom_factory_0000>, <&macaddr_factory_e000>;
		nvmem-cell-names = "eeprom", "mac-address";
	};
};

&pcie1 {
	wifi1: wifi@0,0 {
		compatible = "pci14c3,7662";
		reg = <0x0000 0 0 0 0>;
		nvmem-cells = <&eeprom_factory_8000>, <&macaddr_factory_e007>;
		nvmem-cell-names = "eeprom", "mac-address";
		ieee80211-freq-limit = <5000000 6000000>;
	};
};

&pcie2 {
	status = "disabled";
};

&gmac0 {
	nvmem-cells = <&macaddr_factory_e000>;
	nvmem-cell-names = "mac-address";
};

&gmac1 {
	status = "okay";
	label = "wan";
	phy-mode = "rgmii";
	phy-handle = <&ethphy4>;
	nvmem-cells = <&macaddr_factory_e000>;
	nvmem-cell-names = "mac-address";
};

&ethphy4 {
	/delete-property/ interrupts;
};

&switch0 {
	status = "okay";

	ports {
		port@0 {
			status = "okay";
			label = "lan0";
		};

		port@1 {
			status = "okay";
			label = "lan1";
		};

		port@2 {
			status = "okay";
			label = "lan2";
		};

		port@3 {
			status = "okay";
			label = "lan3";
		};

		port@6 {
			phy-mode = "trgmii";
		};
	};
};

&state_default {
	gpio {
		groups = "wdt", "jtag", "uart3";
		function = "gpio";
	};
};

&i2c {
	status = "disabled";
};

I also tried to remove gmac1 setup from the device tree and add wan label and references to macaddr phandle to port@4. It has absolutely no effect though.

devicetree

/* Commented out
&gmac1 {
	status = "okay";
	label = "wan";
	phy-handle = <&ethphy4>;
	nvmem-cells = <&macaddr_factory_e007>;
	nvmem-cell-names = "mac-address";
	*/
};

&ethphy4 {
	/delete-property/ interrupts;
};
*/

&switch0 {
	status = "okay";

	ports {
		... /* This part is not changed */

		port@4 {
			status = "okay";
			label = "wan";

			nvmem-cells = <&macaddr_factory_e007>;
			nvmem-cell-names = "mac-address";
		};
	};
};

The output I get from ifstatus wan:

wan

{
	"up": false,
	"pending": false,
	"available": false,
	"autostart": true,
	"dynamic": false,
	"proto": "dhcp",
	"device": "br-wan",
	"data": {
		
	},
	"errors": [
		{
			"subsystem": "interface",
			"code": "NO_DEVICE"
		}
	]
}

My versions:

OpenWrt v23.05.5
U-Boot v2024.10

123serge123 · November 4, 2024, 3:53pm

At least I see one error in your config: gmac0(eth0) and switch port 6 must have the same link mode. In your case:

and

they are different.

glebzlat · November 5, 2024, 4:56am

Removed. No effect

glebzlat · November 5, 2024, 10:02am

What can be a reason for this? I see that other devices (particularly Elecom WXC X1800gsx, Dlink DIR-853 A1) just define interfaces and that's it. But in my case something is wrong and I don't know where to dig.

jdwl1o1 · November 5, 2024, 9:34pm

As it is custom hardware it is really impossible for an outsider to know how the manufacturer has connected the interfaces. Mediatek provide many, many options for connecting the SOC to external devices and breaking out the internal PCIe bus.
Review the documentation from the manufacturer, look into how their SDK sets up the interfaces and if that doesn't help talk to your account manager for better docs and support.

glebzlat · November 6, 2024, 12:28pm

Ok, PCIe devices are external. But switch is integrated into the SoC. Is there a way to figure out the right configuration for it?

jdwl1o1 · November 6, 2024, 7:42pm

You’ve already tried the ‘basic’ setup other devices are using without success so the logical conclusion is that your hardware has been customised.
Without documentation of what those customisations are you’re flailing in the dark.
You can try the shotgun approach - keep booting initramfs images from similar devices until something works and then copy that devices dts.
Or trace the circuits on the pcb with the hope of identifying what the manufacturer has done