MT7621 PCIe DTS issues. (MT76 stopped working with latest updates)

drbrains · May 20, 2018, 8:06am

Anyone else have similar problems:

[   12.255680] mt7603e 0000:02:00.0: ASIC revision: 76030010
[   12.322172] mt7603e 0000:02:00.0: Firmware Version: ap_pcie
[   12.327774] mt7603e 0000:02:00.0: Build Time: 20160107100755
[   13.397681] MCU message -1 (seq 1) timed out
[   13.401935] mt7603e 0000:02:00.0: Download request failed
[   13.407495] mt7603e: probe of 0000:02:00.0 failed with error -145
[   13.416477] mt76x2e 0000:01:00.0: ASIC revision: 76120044
[   13.448324] mt76x2e 0000:01:00.0: ROM patch build: 20141115060606a
[   13.458043] mt76x2e 0000:01:00.0: Firmware Version: 0.0.00
[   13.463523] mt76x2e 0000:01:00.0: Build: 1
[   13.467597] mt76x2e 0000:01:00.0: Build Time: 201507311614____
[   13.487708] mt76x2e 0000:01:00.0: Firmware running!
[   14.517682] mt76x2e 0000:01:00.0: MCU message 1 (seq 1) timed out
[   14.524895] ieee80211 phy1: Selected rate control algorithm 'minstrel_ht'

MT7603 is not loading at all due firmware missing. The MT7612e not running. Keep getting the same MCU timed out. @nbd, how should I throubleshoot this issue?

nbd · May 20, 2018, 9:25am

Sometimes you need to power cycle the device (reboot is not enough). Did you try that?

drbrains · May 20, 2018, 9:40am

I’m quite sure I did, but I will try that later when I’m home again. I can understand how that might make a difference for the “firmware not downloading”, but that MT7612 is installing the firmware and that one doesn’t work either. The problem started since the 2018-5-15 revision and the 5-18 update didn’t help.

I did a full fresh “git clone” and package install. Keep everything “vanilla” except adding “Luci” to the build and I compiled everything from source.

sergachev · May 21, 2018, 8:55pm

Same problem here with the latest build.

nbd · May 22, 2018, 10:36am

Does the latest version from my staging tree help?

drbrains · May 22, 2018, 11:11am

compiling now with your 2018-5-20 version. I did a "make clean" to be sure I have as vanilla as possible. It will take a while, I will report back.

drbrains · May 22, 2018, 11:50am

Unfortunately the latest version doesn't seem to help

[   12.231715] mt7603e 0000:02:00.0: ASIC revision: 76030010
[   12.298611] mt7603e 0000:02:00.0: Firmware Version: ap_pcie
[   12.304185] mt7603e 0000:02:00.0: Build Time: 20160107100755
[   13.358058] MCU message -1 (seq 1) timed out
[   13.362314] mt7603e 0000:02:00.0: Download request failed
[   13.367871] mt7603e: probe of 0000:02:00.0 failed with error -145
[   13.377026] mt76x2e 0000:01:00.0: ASIC revision: 76120044
[   13.408703] mt76x2e 0000:01:00.0: ROM patch build: 20141115060606a
[   13.418453] mt76x2e 0000:01:00.0: Firmware Version: 0.0.00
[   13.423934] mt76x2e 0000:01:00.0: Build: 1
[   13.428007] mt76x2e 0000:01:00.0: Build Time: 201507311614____
[   13.448116] mt76x2e 0000:01:00.0: Firmware running!
[   14.478060] mt76x2e 0000:01:00.0: MCU message 1 (seq 1) timed out
[   14.485186] ieee80211 phy1: Selected rate control algorithm 'minstrel_ht'
[   14.494094] PPP generic driver version 2.4.2
[   14.499976] NET: Registered protocol family 24
[   14.508572] kmodloader: done loading kernel modules from /etc/modules.d/*
[   22.782097] mtk_soc_eth 1e100000.ethernet: PPE started
[   22.797708] br-lan: port 1(eth0.1) entered blocking state
[   22.803179] br-lan: port 1(eth0.1) entered disabled state
[   22.809842] device eth0.1 entered promiscuous mode
[   22.814729] device eth0 entered promiscuous mode
[   22.824908] br-lan: port 1(eth0.1) entered blocking state
[   22.830357] br-lan: port 1(eth0.1) entered forwarding state
[   22.836360] IPv6: ADDRCONF(NETDEV_UP): br-lan: link is not ready
[   23.082221] jffs2_scan_eraseblock(): End of filesystem marker found at 0x0
[   23.089500] jffs2_build_filesystem(): unlocking the mtd device... 
[   23.089627] done.
[   23.097854] jffs2_build_filesystem(): erasing all blocks after the end marker... 
[   23.838615] IPv6: ADDRCONF(NETDEV_CHANGE): br-lan: link becomes ready
[   59.118712] done.
[   59.120684] jffs2: notice: (1216) jffs2_build_xattr_subsystem: complete building xattr subsystem, 0 of xdatum (0 unchecked, 0 orphan) and 0 of xref (0 dead, 0 orphan) found.
[   59.285711] overlayfs: upper fs does not support tmpfile.
[  101.345501] mt76x2e 0000:01:00.0: MCU message 20 (seq 2) timed out

the last line is the "error" after I enabled the MT7612e wifi. The MT7603 does not even show up in Luci.

nbd · May 22, 2018, 11:53am

What's the last version that works for you?

nbd · May 22, 2018, 11:54am

Also, what device are you using?

drbrains · May 22, 2018, 12:11pm

My test device is a ZBT-WE1326. It stopped working with the 2018-5-15 MT76 revision. Before that it worked. Tomorrow I am able to check it on MT7628 SOC also combined with an MT7612e (Mercury MAC1200R-v2).

Its strange that the guys running the DLink 860 didn't report anything yet. I would assume they would run into the same problems. (@Mushoz)

Mushoz · May 22, 2018, 12:24pm

I am running the 18.06 snapshot at the moment. That branch is running slightly older mt76 drivers than the master branch. Not sure which commit specifically breaks mt76 for you? Wifi seems fine for me on 18.06 by the way.

drbrains · May 22, 2018, 1:08pm

So I tried the latest 2018-05-20, downgraded to 05-18, 05-15 and finally to 04-18. The 04-18 version was working before without a doubt. The only thing that might be related to this problem is the PCI(e) driver that had patches 7 days ago (same time the MT76 driver updated to 05-15)

Let me try to update the MT76 to the latest one on an older 4.9 build...

drbrains · May 22, 2018, 2:02pm

MT76-2018-05-20 on 4.9.87:

[    0.000000] Linux version 4.9.87 (drbrains@debian) (gcc version 6.3.0 (OpenWrt GCC 6.3.0 r6038-13e8d54917) ) #0 SMP Sat Mar 17 23:59:18 2018
[    0.000000] SoC Type: MediaTek MT7621 ver:1 eco:3
[    0.000000] bootconsole [early0] enabled
[    0.000000] CPU0 revision is: 0001992f (MIPS 1004Kc)
[    0.000000] MIPS: machine is ZBT-WE1326
[    0.000000] Determined physical RAM map:
[    0.000000]  memory: 1c000000 @ 00000000 (usable)
[    0.000000]  memory: 04000000 @ 20000000 (usable)
[    0.000000] Initrd not found or empty - disabling initrd
[    0.000000] VPE topology {2,2} total 4
[    0.000000] Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
[    0.000000] Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
[    0.000000] MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
[    0.000000] Zone ranges:
[    0.000000]   Normal   [mem 0x0000000000000000-0x000000000fffffff]
[    0.000000]   HighMem  [mem 0x0000000010000000-0x0000000023ffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000000000-0x000000001bffffff]
[    0.000000]   node   0: [mem 0x0000000020000000-0x0000000023ffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x0000000023ffffff]
[    0.000000] On node 0 totalpages: 131072
[    0.000000] free_area_init_node: node 0, pgdat 804de4c0, node_mem_map 81003000
[    0.000000]   Normal zone: 512 pages used for memmap
[    0.000000]   Normal zone: 0 pages reserved
[    0.000000]   Normal zone: 65536 pages, LIFO batch:15
[    0.000000]   HighMem zone: 65536 pages, LIFO batch:15
[    0.000000] percpu: Embedded 12 pages/cpu @8148e000 s17744 r8192 d23216 u49152
[    0.000000] pcpu-alloc: s17744 r8192 d23216 u49152 alloc=12*4096
[    0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 130560
[    0.000000] Kernel command line: console=ttyS0,115200 rootfstype=squashfs,jffs2
[    0.000000] PID hash table entries: 1024 (order: 0, 4096 bytes)
[    0.000000] Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
[    0.000000] Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
[    0.000000] Writing ErrCtl register=00070c40
[    0.000000] Readback ErrCtl register=00070c40
[    0.000000] Memory: 513676K/524288K available (3883K kernel code, 212K rwdata, 888K rodata, 252K init, 267K bss, 10612K reserved, 0K cma-reserved, 262144K highmem)
[    0.000000] SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000] NR_IRQS:256
[    0.000000] clocksource: GIC: mask: 0xffffffffffffffff max_cycles: 0xcaf478abb4, max_idle_ns: 440795247997 ns
[    0.000000] clocksource: MIPS: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 4343773742 ns
[    0.000010] sched_clock: 32 bits at 440MHz, resolution 2ns, wraps every 4880645118ns
[    0.007798] Calibrating delay loop... 586.13 BogoMIPS (lpj=2930688)
[    0.070416] pid_max: default: 32768 minimum: 301
[    0.075150] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
[    0.081676] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
[    0.093210] Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
[    0.093223] Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
[    0.093235] MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
[    0.093403] CPU1 revision is: 0001992f (MIPS 1004Kc)
[    0.177390] Synchronize counters for CPU 1: 
[    0.177392] done.
[    0.194451] Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
[    0.194459] Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
[    0.194468] MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
[    0.194560] CPU2 revision is: 0001992f (MIPS 1004Kc)
[    0.269423] Synchronize counters for CPU 2: 
[    0.269424] done.
[    0.277083] Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
[    0.277090] Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
[    0.277099] MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
[    0.277195] CPU3 revision is: 0001992f (MIPS 1004Kc)
[    0.354603] Synchronize counters for CPU 3: 
[    0.354604] done.
[    0.360839] Brought up 4 CPUs
[    0.367516] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.377301] futex hash table entries: 1024 (order: 3, 32768 bytes)
[    0.383562] pinctrl core: initialized pinctrl subsystem
[    0.389376] NET: Registered protocol family 16
[    0.402768] FPU Affinity set after 11720 emulations
[    0.429156] pull PCIe RST: RALINK_RSTCTRL = 0
[    0.733800] release PCIe RST: RALINK_RSTCTRL = 7000000
[    0.738833] ***** Xtal 40MHz *****
[    0.742222] release PCIe RST: RALINK_RSTCTRL = 7000000
[    0.747299] Port 0 N_FTS = 1b102800
[    0.750757] Port 1 N_FTS = 1b105000
[    0.754230] Port 2 N_FTS = 1b105000
[    1.908825] PCIE0 no card, disable it(RST&CLK)
[    1.913167]  -> 10207f2
[    1.915601] PCIE1 enabled
[    1.918174] PCIE2 enabled
[    1.920773] PCI host bridge /pcie@1e140000 ranges:
[    1.925537]  MEM 0x0000000060000000..0x000000006fffffff
[    1.930726]   IO 0x000000001e160000..0x000000001e16ffff
[    1.935891] PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
[    1.974385] mt7621_gpio 1e000600.gpio: registering 32 gpios
[    1.980278] mt7621_gpio 1e000600.gpio: registering 32 gpios
[    1.986046] mt7621_gpio 1e000600.gpio: registering 32 gpios
[    1.993071] PCI host bridge to bus 0000:00
[    1.997100] pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
[    2.003943] pci_bus 0000:00: root bus resource [io  0xffffffff]
[    2.009788] pci_bus 0000:00: root bus resource [??? 0x00000000 flags 0x0]
[    2.016542] pci_bus 0000:00: No busn resource found for root bus, will use [bus 00-ff]
[    2.024446] pci 0000:00:00.0: [0e8d:0801] type 01 class 0x060400
[    2.024479] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x7fffffff]
[    2.024492] pci 0000:00:00.0: reg 0x14: [mem 0x60300000-0x6030ffff]
[    2.024555] pci 0000:00:00.0: supports D1
[    2.024564] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[    2.024796] pci 0000:00:01.0: [0e8d:0801] type 01 class 0x060400
[    2.024822] pci 0000:00:01.0: reg 0x10: [mem 0x00000000-0x7fffffff]
[    2.024836] pci 0000:00:01.0: reg 0x14: [mem 0x60310000-0x6031ffff]
[    2.024886] pci 0000:00:01.0: supports D1
[    2.024895] pci 0000:00:01.0: PME# supported from D0 D1 D3hot
[    2.025301] pci 0000:01:00.0: [14c3:7662] type 00 class 0x028000
[    2.025339] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x000fffff 64bit]
[    2.025386] pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0000ffff pref]
[    2.025479] pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
[    2.025667] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
[    2.025864] pci 0000:02:00.0: [14c3:7603] type 00 class 0x028000
[    2.025895] pci 0000:02:00.0: reg 0x10: [mem 0x00000000-0x000fffff]
[    2.026015] pci 0000:02:00.0: PME# supported from D0 D3hot D3cold
[    2.026204] pci_bus 0000:02: busn_res: [bus 02-ff] end is updated to 02
[    2.026223] pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to 02
[    2.026281] pci 0000:00:00.0: BAR 0: no space for [mem size 0x80000000]
[    2.032822] pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x80000000]
[    2.039709] pci 0000:00:01.0: BAR 0: no space for [mem size 0x80000000]
[    2.046284] pci 0000:00:01.0: BAR 0: failed to assign [mem size 0x80000000]
[    2.053188] pci 0000:00:00.0: BAR 8: assigned [mem 0x60000000-0x600fffff]
[    2.059939] pci 0000:00:00.0: BAR 9: assigned [mem 0x60100000-0x601fffff pref]
[    2.067097] pci 0000:00:01.0: BAR 8: assigned [mem 0x60200000-0x602fffff]
[    2.073848] pci 0000:00:00.0: BAR 1: assigned [mem 0x60300000-0x6030ffff]
[    2.080577] pci 0000:00:01.0: BAR 1: assigned [mem 0x60310000-0x6031ffff]
[    2.087335] pci 0000:01:00.0: BAR 0: assigned [mem 0x60000000-0x600fffff 64bit]
[    2.094584] pci 0000:01:00.0: BAR 6: assigned [mem 0x60100000-0x6010ffff pref]
[    2.101762] pci 0000:00:00.0: PCI bridge to [bus 01]
[    2.106670] pci 0000:00:00.0:   bridge window [mem 0x60000000-0x600fffff]
[    2.113418] pci 0000:00:00.0:   bridge window [mem 0x60100000-0x601fffff pref]
[    2.120580] pci 0000:02:00.0: BAR 0: assigned [mem 0x60200000-0x602fffff]
[    2.127330] pci 0000:00:01.0: PCI bridge to [bus 02]
[    2.132239] pci 0000:00:01.0:   bridge window [mem 0x60200000-0x602fffff]
[    2.139008] BAR0 at slot 0 = 0
[    2.141996] bus=0x0, slot = 0x0
[    2.145115] BAR0 at slot 1 = 0
[    2.148130] bus=0x0, slot = 0x1
[    2.151263] bus=0x1, slot = 0x0, irq=0xff
[    2.155229] bus=0x2, slot = 0x1, irq=0xff
[    2.160528] clocksource: Switched to clocksource GIC
[    2.167073] NET: Registered protocol family 2
[    2.172158] TCP established hash table entries: 2048 (order: 1, 8192 bytes)
[    2.179048] TCP bind hash table entries: 2048 (order: 2, 16384 bytes)
[    2.185467] TCP: Hash tables configured (established 2048 bind 2048)
[    2.191818] UDP hash table entries: 256 (order: 1, 8192 bytes)
[    2.197582] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
[    2.204062] NET: Registered protocol family 1
[    2.208386] PCI: CLS 80 bytes, default 32
[    2.440469] 4 CPUs re-calibrate udelay(lpj = 2924544)
[    2.446719] Crashlog allocated RAM at address 0x3f00000
[    2.452297] workingset: timestamp_bits=30 max_order=17 bucket_order=0
[    2.465198] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    2.471022] jffs2: version 2.2 (NAND) (SUMMARY) (LZMA) (RTIME) (CMODE_PRIORITY) (c) 2001-2006 Red Hat, Inc.
[    2.482371] random: fast init done
[    2.486790] bounce: pool size: 64 pages
[    2.490595] io scheduler noop registered
[    2.494421] io scheduler deadline registered (default)
[    2.500451] Serial: 8250/16550 driver, 3 ports, IRQ sharing disabled
[    2.508065] console [ttyS0] disabled
[    2.511638] 1e000c00.uartlite: ttyS0 at MMIO 0x1e000c00 (irq = 18, base_baud = 3125000) is a 16550A
[    2.520610] console [ttyS0] enabled
[    2.527471] bootconsole [early0] disabled
[    2.536705] MediaTek Nand driver init, version v2.1 Fix AHB virt2phys error
[    2.544053] spi-mt7621 1e000b00.spi: sys_freq: 50000000
[    2.560730] m25p80 spi32766.0: using chunked io (size=32)
[    2.566126] m25p80 spi32766.0: w25q128 (16384 Kbytes)
[    2.571216] 4 fixed-partitions partitions found on MTD device spi32766.0
[    2.577883] Creating 4 MTD partitions on "spi32766.0":
[    2.583031] 0x000000000000-0x000000030000 : "u-boot"
[    2.589322] 0x000000030000-0x000000040000 : "u-boot-env"
[    2.595990] 0x000000040000-0x000000050000 : "factory"
[    2.602443] 0x000000050000-0x000001000000 : "firmware"
[    2.635847] 2 uimage-fw partitions found on MTD device firmware
[    2.641798] 0x000000050000-0x0000001e464b : "kernel"
[    2.647963] 0x0000001e464b-0x000001000000 : "rootfs"
[    2.654091] mtd: device 5 (rootfs) set to be root filesystem
[    2.659831] 1 squashfs-split partitions found on MTD device rootfs
[    2.666044] 0x000000690000-0x000001000000 : "rootfs_data"
[    2.673053] libphy: Fixed MDIO Bus: probed
[    2.742554] libphy: mdio: probed
[    4.145368] mtk_soc_eth 1e100000.ethernet: loaded mt7530 driver
[    4.152210] mtk_soc_eth 1e100000.ethernet eth0: mediatek frame engine at 0xbe100000, irq 21
[    4.162847] NET: Registered protocol family 10
[    4.168910] NET: Registered protocol family 17
[    4.173489] 8021q: 802.1Q VLAN Support v1.8
[    4.180227] hctosys: unable to open rtc device (rtc0)
[    4.190389] VFS: Mounted root (squashfs filesystem) readonly on device 31:5.
[    4.198078] Freeing unused kernel memory: 252K
[    4.202549] This architecture does not have kernel memory protection.
[    4.324989] mtk_soc_eth 1e100000.ethernet eth0: port 3 link up
[    4.583537] random: crng init done
[    5.007167] init: Console is alive
[    5.010878] init: - watchdog -
[    5.096885] mtk_soc_eth 1e100000.ethernet eth0: port 4 link up
[    6.036296] kmodloader: loading kernel modules from /etc/modules-boot.d/*
[    6.147092] usbcore: registered new interface driver usbfs
[    6.152728] usbcore: registered new interface driver hub
[    6.158191] usbcore: registered new device driver usb
[    6.166721] kmodloader: done loading kernel modules from /etc/modules-boot.d/*
[    6.180850] init: - preinit -
[   10.243360] mount_root: jffs2 not ready yet, using temporary tmpfs overlay
[   10.270192] urandom-seed: Seed file not found (/etc/urandom.seed)
[   10.336907] procd: - early -
[   10.339886] procd: - watchdog -
[   11.010675] procd: - watchdog -
[   11.014116] procd: - ubus -
[   11.155369] procd: - init -
[   11.421776] kmodloader: loading kernel modules from /etc/modules.d/*
[   11.486389] NET: Registered protocol family 38
[   11.500411] device-mapper: ioctl: 4.35.0-ioctl (2016-06-23) initialised: dm-devel@redhat.com
[   11.516255] tun: Universal TUN/TAP device driver, 1.6
[   11.521381] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
[   11.531387] ip6_tables: (C) 2000-2006 Netfilter Core Team
[   11.543384] Netfilter messages via NETLINK v0.30.
[   11.549983] ip_set: protocol 6
[   11.576945] cryptodev: driver 1.9 loaded.
[   11.583926] Loading modules backported from Linux version wt-2017-11-01-0-gfe248fc2c180
[   11.591954] Backport generated by backports.git v4.14-rc2-1-31-g86cf0e5d
[   11.601008] ip_tables: (C) 2000-2006 Netfilter Core Team
[   11.613565] nf_conntrack version 0.5.0 (8192 buckets, 32768 max)
[   11.659912] xt_time: kernel timezone is -0000
[   11.711071] mt7603e 0000:02:00.0: ASIC revision: 76030010
[   11.720008] mt7603e 0000:02:00.0: Firmware Version: ap_pcie
[   11.725614] mt7603e 0000:02:00.0: Build Time: 20160107100755
[   12.130513] firmware init done
[   12.302365] ieee80211 phy0: Selected rate control algorithm 'minstrel_ht'
[   12.310312] mt76x2e 0000:01:00.0: ASIC revision: 76120044
[   12.413878] mt76x2e 0000:01:00.0: ROM patch build: 20141115060606a
[   12.425515] mt76x2e 0000:01:00.0: Firmware Version: 0.0.00
[   12.431027] mt76x2e 0000:01:00.0: Build: 1
[   12.435112] mt76x2e 0000:01:00.0: Build Time: 201507311614____
[   12.460592] mt76x2e 0000:01:00.0: Firmware running!
[   12.467321] ieee80211 phy1: Selected rate control algorithm 'minstrel_ht'
[   12.476252] PPP generic driver version 2.4.2
[   12.482136] NET: Registered protocol family 24
[   12.489514] kmodloader: done loading kernel modules from /etc/modules.d/*
[   21.376156] br-lan: port 1(eth0.1) entered blocking state
[   21.381608] br-lan: port 1(eth0.1) entered disabled state
[   21.387780] device eth0.1 entered promiscuous mode
[   21.392890] device eth0 entered promiscuous mode
[   21.400307] br-lan: port 1(eth0.1) entered blocking state
[   21.405841] br-lan: port 1(eth0.1) entered forwarding state
[   21.411903] IPv6: ADDRCONF(NETDEV_UP): br-lan: link is not ready
[   21.892320] jffs2_scan_eraseblock(): End of filesystem marker found at 0x0
[   21.900637] jffs2_build_filesystem(): unlocking the mtd device... 
[   21.906754] done.
[   21.908779] jffs2_build_filesystem(): erasing all blocks after the end marker... 
[   22.371075] IPv6: ADDRCONF(NETDEV_CHANGE): br-lan: link becomes ready

on 4.14.41:

[    0.000000] Linux version 4.14.41 (drbrains@debian) (gcc version 7.3.0 (OpenWrt GCC 7.3.0 r6953-aa30eb5b07)) #0 SMP Tue May 22 07:27:38 2018
[    0.000000] SoC Type: MediaTek MT7621 ver:1 eco:3
[    0.000000] bootconsole [early0] enabled
[    0.000000] CPU0 revision is: 0001992f (MIPS 1004Kc)
[    0.000000] MIPS: machine is ZBT-WE1326
[    0.000000] Determined physical RAM map:
[    0.000000]  memory: 1c000000 @ 00000000 (usable)
[    0.000000]  memory: 04000000 @ 20000000 (usable)
[    0.000000] Initrd not found or empty - disabling initrd
[    0.000000] VPE topology {2,2} total 4
[    0.000000] Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
[    0.000000] Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
[    0.000000] MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
[    0.000000] Zone ranges:
[    0.000000]   Normal   [mem 0x0000000000000000-0x000000000fffffff]
[    0.000000]   HighMem  [mem 0x0000000010000000-0x0000000023ffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000000000-0x000000001bffffff]
[    0.000000]   node   0: [mem 0x0000000020000000-0x0000000023ffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x0000000023ffffff]
[    0.000000] On node 0 totalpages: 131072
[    0.000000] free_area_init_node: node 0, pgdat 80573b20, node_mem_map 81003000
[    0.000000]   Normal zone: 512 pages used for memmap
[    0.000000]   Normal zone: 0 pages reserved
[    0.000000]   Normal zone: 65536 pages, LIFO batch:15
[    0.000000]   HighMem zone: 65536 pages, LIFO batch:15
[    0.000000] random: get_random_bytes called from start_kernel+0x90/0x4a4 with crng_init=0
[    0.000000] percpu: Embedded 14 pages/cpu @8148f000 s26192 r8192 d22960 u57344
[    0.000000] pcpu-alloc: s26192 r8192 d22960 u57344 alloc=14*4096
[    0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 130560
[    0.000000] Kernel command line: console=ttyS0,115200 rootfstype=squashfs,jffs2
[    0.000000] PID hash table entries: 1024 (order: 0, 4096 bytes)
[    0.000000] Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
[    0.000000] Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
[    0.000000] Writing ErrCtl register=00070c40
[    0.000000] Readback ErrCtl register=00070c40
[    0.000000] Memory: 513060K/524288K available (4382K kernel code, 229K rwdata, 972K rodata, 228K init, 268K bss, 11228K reserved, 0K cma-reserved, 262144K highmem)
[    0.000000] SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000] NR_IRQS: 256
[    0.000000] clocksource: GIC: mask: 0xffffffffffffffff max_cycles: 0xcaf478abb4, max_idle_ns: 440795247997 ns
[    0.000000] clocksource: MIPS: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 4343773742 ns
[    0.000011] sched_clock: 32 bits at 440MHz, resolution 2ns, wraps every 4880645118ns
[    0.007824] Calibrating delay loop... 586.13 BogoMIPS (lpj=2930688)
[    0.073979] pid_max: default: 32768 minimum: 301
[    0.078751] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
[    0.085260] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
[    0.094096] Hierarchical SRCU implementation.
[    0.099195] smp: Bringing up secondary CPUs ...
[    0.105221] Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
[    0.105231] Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
[    0.105244] MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
[    0.105399] CPU1 revision is: 0001992f (MIPS 1004Kc)
[    0.163976] Synchronize counters for CPU 1: done.
[    0.205686] Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
[    0.205696] Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
[    0.205705] MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
[    0.205782] CPU2 revision is: 0001992f (MIPS 1004Kc)
[    0.255244] Synchronize counters for CPU 2: done.
[    0.286480] Primary instruction cache 32kB, VIPT, 4-way, linesize 32 bytes.
[    0.286487] Primary data cache 32kB, 4-way, PIPT, no aliases, linesize 32 bytes
[    0.286495] MIPS secondary cache 256kB, 8-way, linesize 32 bytes.
[    0.286579] CPU3 revision is: 0001992f (MIPS 1004Kc)
[    0.340426] Synchronize counters for CPU 3: done.
[    0.370287] smp: Brought up 1 node, 4 CPUs
[    0.377826] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.387618] futex hash table entries: 1024 (order: 3, 32768 bytes)
[    0.393938] pinctrl core: initialized pinctrl subsystem
[    0.399705] NET: Registered protocol family 16
[    0.413778] FPU Affinity set after 11720 emulations
[    0.714823] ***** Xtal 40MHz *****
[    0.720291] Port 0 N_FTS = 1b102800
[    0.723684] Port 1 N_FTS = 1b105000
[    0.727138] Port 2 N_FTS = 1b105000
[    1.781971] PCIE0 no card, disable it(RST&CLK)
[    1.786316] PCIE1 enabled
[    1.788905] PCIE2 enabled
[    1.791533] PCI host bridge /pcie@1e140000 ranges:
[    1.796268]  MEM 0x0000000060000000..0x000000006fffffff
[    1.801438]   IO 0x000000001e160000..0x000000001e16ffff
[    1.806639] PCI coherence region base: 0x60000000, mask/settings: 0xf0000002
[    1.821425] mt7621_gpio 1e000600.gpio: registering 32 gpios
[    1.827228] mt7621_gpio 1e000600.gpio: registering 32 gpios
[    1.832938] mt7621_gpio 1e000600.gpio: registering 32 gpios
[    1.839811] PCI host bridge to bus 0000:00
[    1.843851] pci_bus 0000:00: root bus resource [mem 0x60000000-0x6fffffff]
[    1.850688] pci_bus 0000:00: root bus resource [io  0xffffffff]
[    1.856534] pci_bus 0000:00: root bus resource [??? 0x00000000 flags 0x0]
[    1.863290] pci_bus 0000:00: No busn resource found for root bus, will use [bus 00-ff]
[    1.871183] pci 0000:00:00.0: [0e8d:0801] type 01 class 0x060400
[    1.871227] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x7fffffff]
[    1.871239] pci 0000:00:00.0: reg 0x14: [mem 0x60300000-0x6030ffff]
[    1.871306] pci 0000:00:00.0: supports D1
[    1.871315] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[    1.871572] pci 0000:00:01.0: [0e8d:0801] type 01 class 0x060400
[    1.871602] pci 0000:00:01.0: reg 0x10: [mem 0x00000000-0x7fffffff]
[    1.871617] pci 0000:00:01.0: reg 0x14: [mem 0x60310000-0x6031ffff]
[    1.871668] pci 0000:00:01.0: supports D1
[    1.871676] pci 0000:00:01.0: PME# supported from D0 D1 D3hot
[    1.872104] pci 0000:01:00.0: [14c3:7662] type 00 class 0x028000
[    1.872160] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x000fffff 64bit]
[    1.872205] pci 0000:01:00.0: reg 0x30: [mem 0x00000000-0x0000ffff pref]
[    1.872300] pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
[    1.872467] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
[    1.872669] pci 0000:02:00.0: [14c3:7603] type 00 class 0x028000
[    1.872716] pci 0000:02:00.0: reg 0x10: [mem 0x00000000-0x000fffff]
[    1.872844] pci 0000:02:00.0: PME# supported from D0 D3hot D3cold
[    1.873014] pci_bus 0000:02: busn_res: [bus 02-ff] end is updated to 02
[    1.873035] pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to 02
[    1.873107] pci 0000:00:00.0: BAR 0: no space for [mem size 0x80000000]
[    1.879647] pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x80000000]
[    1.886540] pci 0000:00:01.0: BAR 0: no space for [mem size 0x80000000]
[    1.893109] pci 0000:00:01.0: BAR 0: failed to assign [mem size 0x80000000]
[    1.900011] pci 0000:00:00.0: BAR 8: assigned [mem 0x60000000-0x600fffff]
[    1.906768] pci 0000:00:00.0: BAR 9: assigned [mem 0x60100000-0x601fffff pref]
[    1.913925] pci 0000:00:01.0: BAR 8: assigned [mem 0x60200000-0x602fffff]
[    1.920674] pci 0000:00:00.0: BAR 1: assigned [mem 0x60300000-0x6030ffff]
[    1.927411] pci 0000:00:01.0: BAR 1: assigned [mem 0x60310000-0x6031ffff]
[    1.934162] pci 0000:01:00.0: BAR 0: assigned [mem 0x60000000-0x600fffff 64bit]
[    1.941404] pci 0000:01:00.0: BAR 6: assigned [mem 0x60100000-0x6010ffff pref]
[    1.948583] pci 0000:00:00.0: PCI bridge to [bus 01]
[    1.953496] pci 0000:00:00.0:   bridge window [mem 0x60000000-0x600fffff]
[    1.960242] pci 0000:00:00.0:   bridge window [mem 0x60100000-0x601fffff pref]
[    1.967414] pci 0000:02:00.0: BAR 0: assigned [mem 0x60200000-0x602fffff]
[    1.974161] pci 0000:00:01.0: PCI bridge to [bus 02]
[    1.979064] pci 0000:00:01.0:   bridge window [mem 0x60200000-0x602fffff]
[    1.987141] clocksource: Switched to clocksource GIC
[    1.993504] NET: Registered protocol family 2
[    1.998557] TCP established hash table entries: 2048 (order: 1, 8192 bytes)
[    2.005441] TCP bind hash table entries: 2048 (order: 2, 16384 bytes)
[    2.011895] TCP: Hash tables configured (established 2048 bind 2048)
[    2.018325] UDP hash table entries: 256 (order: 1, 8192 bytes)
[    2.024092] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
[    2.030586] NET: Registered protocol family 1
[    2.034915] PCI: CLS 80 bytes, default 32
[    2.267080] 4 CPUs re-calibrate udelay(lpj = 2924544)
[    2.273287] Crashlog allocated RAM at address 0x3f00000
[    2.278700] workingset: timestamp_bits=30 max_order=17 bucket_order=0
[    2.291185] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    2.296930] jffs2: version 2.2 (NAND) (SUMMARY) (LZMA) (RTIME) (CMODE_PRIORITY) (c) 2001-2006 Red Hat, Inc.
[    2.310039] bounce: pool size: 64 pages
[    2.313814] io scheduler noop registered
[    2.317759] io scheduler deadline registered (default)
[    2.322872] random: fast init done
[    2.327034] Serial: 8250/16550 driver, 3 ports, IRQ sharing disabled
[    2.334592] console [ttyS0] disabled
[    2.338183] 1e000c00.uartlite: ttyS0 at MMIO 0x1e000c00 (irq = 18, base_baud = 3125000) is a 16550A
[    2.347180] console [ttyS0] enabled
[    2.354031] bootconsole [early0] disabled
[    2.362596] cacheinfo: Failed to find cpu0 device node
[    2.367769] cacheinfo: Unable to detect cache hierarchy for CPU 0
[    2.374492] MediaTek Nand driver init, version v2.1 Fix AHB virt2phys error
[    2.382004] spi-mt7621 1e000b00.spi: sys_freq: 50000000
[    2.398594] m25p80 spi0.0: w25q128 (16384 Kbytes)
[    2.403342] 4 fixed-partitions partitions found on MTD device spi0.0
[    2.409711] Creating 4 MTD partitions on "spi0.0":
[    2.414492] 0x000000000000-0x000000030000 : "u-boot"
[    2.420449] 0x000000030000-0x000000040000 : "u-boot-env"
[    2.426594] 0x000000040000-0x000000050000 : "factory"
[    2.432535] 0x000000050000-0x000001000000 : "firmware"
[    2.476238] 2 uimage-fw partitions found on MTD device firmware
[    2.482190] 0x000000050000-0x000000214310 : "kernel"
[    2.488134] 0x000000214310-0x000001000000 : "rootfs"
[    2.493897] mtd: device 5 (rootfs) set to be root filesystem
[    2.499674] 1 squashfs-split partitions found on MTD device rootfs
[    2.505838] 0x000000480000-0x000001000000 : "rootfs_data"
[    2.512737] libphy: Fixed MDIO Bus: probed
[    2.589152] libphy: mdio: probed
[    3.992019] mtk_soc_eth 1e100000.ethernet: loaded mt7530 driver
[    3.998689] mtk_soc_eth 1e100000.ethernet eth0: mediatek frame engine at 0xbe100000, irq 21
[    4.009439] NET: Registered protocol family 10
[    4.015483] Segment Routing with IPv6
[    4.019325] NET: Registered protocol family 17
[    4.023804] 8021q: 802.1Q VLAN Support v1.8
[    4.030544] hctosys: unable to open rtc device (rtc0)
[    4.042397] VFS: Mounted root (squashfs filesystem) readonly on device 31:5.
[    4.050040] Freeing unused kernel memory: 228K
[    4.054477] This architecture does not have kernel memory protection.
[    4.171674] mtk_soc_eth 1e100000.ethernet eth0: port 3 link up
[    4.316639] mtk_soc_eth 1e100000.ethernet eth0: port 4 link up
[    4.417981] random: crng init done
[    4.675808] init: Console is alive
[    4.679537] init: - watchdog -
[    5.489123] kmodloader: loading kernel modules from /etc/modules-boot.d/*
[    5.615081] usbcore: registered new interface driver usbfs
[    5.620717] usbcore: registered new interface driver hub
[    5.626162] usbcore: registered new device driver usb
[    5.641193] xhci-mtk 1e1c0000.xhci: xHCI Host Controller
[    5.646543] xhci-mtk 1e1c0000.xhci: new USB bus registered, assigned bus number 1
[    5.657350] xhci-mtk 1e1c0000.xhci: hcc params 0x01401198 hci version 0x96 quirks 0x00210010
[    5.665833] xhci-mtk 1e1c0000.xhci: irq 20, io mem 0x1e1c0000
[    5.672597] hub 1-0:1.0: USB hub found
[    5.676421] hub 1-0:1.0: 2 ports detected
[    5.681028] xhci-mtk 1e1c0000.xhci: xHCI Host Controller
[    5.686342] xhci-mtk 1e1c0000.xhci: new USB bus registered, assigned bus number 2
[    5.694019] usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
[    5.702985] hub 2-0:1.0: USB hub found
[    5.706808] hub 2-0:1.0: 1 port detected
[    5.780676] kmodloader: done loading kernel modules from /etc/modules-boot.d/*
[    5.797500] init: - preinit -
[    6.585881] mtk_soc_eth 1e100000.ethernet: PPE started
[    9.810118] mount_root: jffs2 not ready yet, using temporary tmpfs overlay
[    9.837804] urandom-seed: Seed file not found (/etc/urandom.seed)
[    9.923919] mtk_soc_eth 1e100000.ethernet: 0x100 = 0x5c60000c, 0x10c = 0x80818
[    9.939287] procd: - early -
[    9.942274] procd: - watchdog -
[   10.597273] procd: - watchdog -
[   10.600725] procd: - ubus -
[   10.785285] procd: - init -
[   11.019889] kmodloader: loading kernel modules from /etc/modules.d/*
[   11.030545] ip6_tables: (C) 2000-2006 Netfilter Core Team
[   11.383810] Loading modules backported from Linux version wt-2017-11-01-0-gfe248fc2c180
[   11.391842] Backport generated by backports.git v4.14-rc2-1-31-g86cf0e5d
[   11.400568] ip_tables: (C) 2000-2006 Netfilter Core Team
[   11.762083] nf_conntrack version 0.5.0 (8192 buckets, 32768 max)
[   11.984464] xt_time: kernel timezone is -0000
[   12.210900] mt7603e 0000:02:00.0: ASIC revision: 76030010
[   12.277831] mt7603e 0000:02:00.0: Firmware Version: ap_pcie
[   12.283407] mt7603e 0000:02:00.0: Build Time: 20160107100755
[   13.317119] MCU message -1 (seq 1) timed out
[   13.321375] mt7603e 0000:02:00.0: Download request failed
[   13.326929] mt7603e: probe of 0000:02:00.0 failed with error -145
[   13.336070] mt76x2e 0000:01:00.0: ASIC revision: 76120044
[   13.367797] mt76x2e 0000:01:00.0: ROM patch build: 20141115060606a
[   13.377693] mt76x2e 0000:01:00.0: Firmware Version: 0.0.00
[   13.383174] mt76x2e 0000:01:00.0: Build: 1
[   13.387273] mt76x2e 0000:01:00.0: Build Time: 201507311614____
[   13.407126] mt76x2e 0000:01:00.0: Firmware running!
[   14.437112] mt76x2e 0000:01:00.0: MCU message 1 (seq 1) timed out
[   14.444355] ieee80211 phy1: Selected rate control algorithm 'minstrel_ht'
[   14.453208] PPP generic driver version 2.4.2
[   14.459023] NET: Registered protocol family 24
[   14.467054] kmodloader: done loading kernel modules from /etc/modules.d/*
[   22.900592] mtk_soc_eth 1e100000.ethernet: PPE started
[   22.918587] br-lan: port 1(eth0.1) entered blocking state
[   22.924003] br-lan: port 1(eth0.1) entered disabled state
[   22.930109] device eth0.1 entered promiscuous mode
[   22.934927] device eth0 entered promiscuous mode
[   22.942875] br-lan: port 1(eth0.1) entered blocking state
[   22.948313] br-lan: port 1(eth0.1) entered forwarding state
[   22.954318] IPv6: ADDRCONF(NETDEV_UP): br-lan: link is not ready
[   23.033817] jffs2_scan_eraseblock(): End of filesystem marker found at 0x0
[   23.041643] jffs2_build_filesystem(): unlocking the mtd device... 
[   23.041744] done.
[   23.049984] jffs2_build_filesystem(): erasing all blocks after the end marker... 
[   23.958095] IPv6: ADDRCONF(NETDEV_CHANGE): br-lan: link becomes ready

Seems the output about the PCI is slightly different. This maybe because of a newer driver, but that also might be the cause of the problem.??
This seems missing:

[    2.139008] BAR0 at slot 0 = 0
[    2.141996] bus=0x0, slot = 0x0
[    2.145115] BAR0 at slot 1 = 0
[    2.148130] bus=0x0, slot = 0x1
[    2.151263] bus=0x1, slot = 0x0, irq=0xff
[    2.155229] bus=0x2, slot = 0x1, irq=0xff

slthomason · May 23, 2018, 2:16pm

We are also having this issue with multiple mt76 devices: https://github.com/openwrt/mt76/issues/173

drbrains · May 23, 2018, 2:35pm

Seems it’s not the MT76 part that’s generated the problem. Using the latest version on 4.9 works just fine. It worked fine on 4.14 until
about 8 days ago. Not sure what was patched since then. I know there were some changes
for the PCI(e) drivers, but maybe the 4.14 revision had some new stuff that did not get (properly) patches.

If I have time I will try the latest a MT76 on the 18.6 version and not on the latest “trunk” / snapshot to see if that makes a difference.

@nbd, sorry for blaming the driver. Hopefully we
can figure out why it’s not working anymore on the latest 4.14 kernel revision

slthomason · May 23, 2018, 9:44pm

We are baffled as well - and blocked in our project as we are on 4.14 kernel. Any luck determining which commit broke it?

Mushoz · May 24, 2018, 7:09am

Shouldn't this be relatively easy to find the commit that is causing this by doing a git bisect? Knowing exactly which commit is causing it, will make it much easier to fix this.

drbrains · May 24, 2018, 1:37pm

That would mean going every commit for the last 10 days or so and possibly have to compile an image every time to try. I'm not so sure we could just spot this problem from the commit "diff". I will give it a try tomorrow afternoon, if I can find the time

dissent1 · May 24, 2018, 1:45pm

Have you tried reverting?
https://git.openwrt.org/?p=openwrt/openwrt.git;a=commit;h=e07baec9faf487fd143976636025b5da55e13c20

dissent1 · May 24, 2018, 2:43pm

at first glance that commit along with cleanups removes this code in probe function:

-       int i;
-
-       for (i = 0; i < 3; i++)
-               pcie_irq[i] = irq_of_parse_and_map(pdev->dev.of_node, i);