Edgerouter ER10x and kernel_e50

Grommish · May 11, 2021, 3:03am

I've been working on getting OpenWrt support for the ER10x. I've got some/most of it working (Switch Setup for Edgerouter 10X (RTL8367RB)), but decided to see if I couldn't get a Proof of Concept build to verify the drivers for everything are actually in the Ubiquiti Kernel (and not in their userspace stuff I can't access).

To do this, I've grabbed the kernel_e50 repo and set it as an external kernel for OpenWrt 19.07. The e50 kernel is based on 4.14, as is 19.07.

I've built it and it compiles (finally), but when I flash it, it unpacks and then just dies.

Any ideas on where to go next? Log below.

And yes, I am well aware this isn't an "OpenWrt" issue, but there are many smart folks here

===================================================================
                MT7621   stage1 code 10:33:11 (ASIC)
                CPU=50000000 HZ BUS=16666666 HZ
==================================================================
Change MPLL source from XTAL to CR...
do MEMPLL setting..
MEMPLL Config : 0x11100000
3PLL mode + External loopback
=== XTAL-40Mhz === DDR-1200Mhz ===
PLL3 FB_DL: 0x8, 1/0 = 586/438 21000000
PLL4 FB_DL: 0x13, 1/0 = 686/338 4D000000
PLL2 FB_DL: 0x16, 1/0 = 687/337 59000000
do DDR setting..[00320381]
Apply DDR3 Setting...(use customer AC)
          0    8   16   24   32   40   48   56   64   72   80   88   96  104  112  120
      --------------------------------------------------------------------------------
0000:|    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
0001:|    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
0002:|    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
0003:|    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
0004:|    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
0005:|    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
0006:|    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
0007:|    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
0008:|    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
0009:|    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
000A:|    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
000B:|    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
000C:|    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
000D:|    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    1
000E:|    0    0    0    0    0    0    0    0    0    1    1    1    1    1    1    1
000F:|    0    0    0    0    0    1    1    1    1    1    1    1    1    1    0    0
0010:|    1    1    1    1    1    1    1    1    1    0    0    0    0    0    0    0
0011:|    1    1    1    1    0    0    0    0    0    0    0    0    0    0    0    0
0012:|    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
0013:|    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
0014:|    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
0015:|    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
0016:|    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
0017:|    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
0018:|    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
0019:|    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
001A:|    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
001B:|    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
001C:|    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
001D:|    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
001E:|    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
001F:|    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0    0
rank 0 coarse = 16
rank 0 fine = 32
B:|    0    0    0    0    0    0    0    1    1    1    0    0    0    0    0    0
opt_dle value:8
DRAMC_R0DELDLY[018]=00001B1B
==================================================================
                RX      DQS perbit delay software calibration
==================================================================
1.0-15 bit dq delay value
==================================================================
bit|     0  1  2  3  4  5  6  7  8  9
--------------------------------------
0 |    12 7 10 8 11 6 11 5 7 10
10 |    6 13 6 11 5 9
--------------------------------------

==================================================================
2.dqs window
x=pass dqs delay value (min~max)center
y=0-7bit DQ of every group
input delay:DQS0 =27 DQS1 = 27
==================================================================
bit     DQS0     bit      DQS1
0  (1~54)27  8  (1~50)25
1  (1~48)24  9  (1~50)25
2  (1~49)25  10  (0~48)24
3  (1~50)25  11  (1~52)26
4  (1~54)27  12  (1~50)25
5  (1~50)25  13  (1~54)27
6  (1~52)26  14  (1~50)25
7  (1~52)26  15  (1~53)27
==================================================================
3.dq delay value last
==================================================================
bit|    0  1  2  3  4  5  6  7  8   9
--------------------------------------
0 |    12 10 12 10 11 8 12 6 9 12
10 |    9 14 8 11 7 9
==================================================================
==================================================================
     TX  perbyte calibration
==================================================================
DQS loop = 15, cmp_err_1 = ffff0200
dqs_perbyte_dly.last_dqsdly_pass[0]=15,  finish count=1
DQS loop = 14, cmp_err_1 = ffff0000
dqs_perbyte_dly.last_dqsdly_pass[1]=14,  finish count=2
DQ loop=15, cmp_err_1 = ffff01aa
DQ loop=14, cmp_err_1 = ffff00aa
dqs_perbyte_dly.last_dqdly_pass[1]=14,  finish count=1
DQ loop=13, cmp_err_1 = ffff00aa
DQ loop=12, cmp_err_1 = ffff0000
dqs_perbyte_dly.last_dqdly_pass[0]=12,  finish count=2
byte:0, (DQS,DQ)=(9,8)
byte:1, (DQS,DQ)=(8,8)
20,data:89
[EMI] DRAMC calibration passed

===================================================================
                MT7621   stage1 code done
                CPU=50000000 HZ BUS=16666666 HZ
===================================================================


U-Boot 1.1.3 (UBNT Build Version: e55_001_8fcb8) (Nov 22 2018 - 09:50:47)

Board: Ralink APSoC DRAM:  256 MB
relocate_code Pointer at: 8ffac000

Config XHCI 40M PLL
Allocate 16 byte aligned buffer: 8ffe0040
Enable NFI Clock
# MTK NAND # : Use HW ECC
NAND ID [C2 DC 90 95 56]
Device found in MTK table, ID: c2dc, EXT_ID: 909556
Support this Device in MTK table! c2dc
select_chip
[NAND]select ecc bit:4, sparesize :64 spare_per_sector=16
Signature matched and data read!
load_fact_bbt success 4095
load fact bbt success
[mtk_nand] probe successfully!
mtd->writesize=2048 mtd->oobsize=64,    mtd->erasesize=131072  devinfo.iowidth=8
..
UBNT e55 13-02555-19 74ACB9EB4375
UBNT BD type=e55, mac=74ACB9EB4375, s/n=74ACB9EB4375, mrev=19, k_idx=0
============================================
Ralink UBoot Version: 4.3.S.0
--------------------------------------------
ASIC MT7621A DualCore (MAC to MT7530 Mode)
DRAM_CONF_FROM: Auto-Detection
DRAM_TYPE: DDR3
DRAM bus: 16 bit
Xtal Mode=3 OCP Ratio=1/3
Flash component: NAND Flash
Date:Nov 22 2018  Time:09:50:47
============================================
icache: sets:256, ways:4, linesz:32 ,total:32768
dcache: sets:256, ways:4, linesz:32 ,total:32768

 ##### The CPU freq = 880 MHZ ####
 estimate memory size =256 Mbytes
#Reset_MT7530
Enable port 0
Disable RTL8367 ports
Please choose the operation:
   1: Load system code to SDRAM via TFTP.
   2: Load system code then write to Flash via TFTP.
   3: Boot system code via Flash (default).
   4: Entr boot command line interface.
   7: Load Boot Loader code then write to Flash via Serial.
   9: Load Boot Loader code then write to Flash via TFTP.
   r: Start TFTP recovery.
default: 3

You choosed 1

 0


1: System Load Linux to SDRAM via TFTP.
 Please Input new ones /or Ctrl-C to discard
        Input device IP (192.168.200.179) ==:
        Input server IP (192.168.200.190) ==:
        Input Linux Kernel filename (openwrt-ramips-mt7621-ubnt-er10x-initramfs-kernel.bin) ==:

 NetTxPacket = 0x8FFE4280
Trying Eth0 (10/100-M)

 Waitting for RX_DMA_BUSY status Start... done


 ETH_STATE_ACTIVE!!
TFTP from server 192.168.200.190; our IP address is 192.168.200.179
Filename 'openwrt-ramips-mt7621-ubnt-er10x-initramfs-kernel.bin'.

 TIMEOUT_COUNT=10,Load address: 0x80a00000
Loading: Got ARP REPLY, set server/gtwy eth addr (24:5e:be:00:00:00)
Got it
#################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         #################################################################
         ##################################################
done
Bytes transferred = 3250285 (31986d hex)
NetBootFileXferSize= 0031986d
Automatic boot of image at addr 0x80A00000 ...
## Booting image at 80a00000 ...
   Image Name:   MIPS OpenWrt Linux-4.14.231
   Image Type:   MIPS Linux Kernel Image (lzma compressed)
   Data Size:    3250221 Bytes =  3.1 MB
   Load Address: 80001000
   Entry Point:  80001000
   Verifying Checksum ... OK
   Uncompressing Kernel Image ... OK
No initrd
## Transferring control to Linux (at address 80001000) ...
>>>>> mips_linux.c, 206: commandline = console=ttyS1,57600n8 ubi.mtd=7 root=ubi0_0 rootfstype=ubifs rootsqimg=squashfs.img rootsqwdir=w rw model=e55 ubnt_ramsize=512
## Giving linux memsize in MB, 256

Starting kernel ...

broly · May 11, 2021, 3:13am

do you have config_early_printk enabled?

i had this issue earlier and i realised it was not enabled.

this would be the reason you're not seeing anything after 'starting kernel'.

btw you're a lucky guy having a u-boot that has a functional 'bootargs' variable. this lantiq board i'm playing with does not. in fact, it does not have much of anything

edit: who's the noob? i'm the noob. CONFIG_MIPS_CMDLINE_BUILTIN_EXTEND=y

Grommish · May 11, 2021, 3:15am

Probably not.. I'll look.. I used the ubnt_er_50_defconfig in the kernel's arch/mips/configs/ as the basis for the build. I'll go look

Grommish · May 11, 2021, 3:18am

In my kernel_50/.config:

CONFIG_SYS_HAS_EARLY_PRINTK=y
CONFIG_OF_EARLY_FLATTREE=y
CONFIG_SERIAL_EARLYCON=y
CONFIG_EARLY_PRINTK=y

and in my generated target/linux/ramips/config-4.14:

CONFIG_EARLY_PRINTK=y
# CONFIG_EARLY_PRINTK_8250 is not set
# CONFIG_EARLY_PRINTK_USB_XDBC is not set
CONFIG_OF_EARLY_FLATTREE=y
CONFIG_SERIAL_EARLYCON=y
# CONFIG_SERIAL_EARLYCON_ARM_SEMIHOST is not set
CONFIG_SYS_HAS_EARLY_PRINTK=y

broly · May 11, 2021, 3:23am

the next question is: do you have the 8250 serial driver enabled?


CONFIG_SERIAL_8250=y
# CONFIG_SERIAL_8250_DEPRECATED_OPTIONS is not set
# CONFIG_SERIAL_8250_FINTEK is not set
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_SERIAL_8250_NR_UARTS=2
CONFIG_SERIAL_8250_RUNTIME_UARTS=2

or something like that.

i don't use kernel 5x so i'm not sure if early_printk_8250 is an option for 4.x. i certainly haven't seen it. but if it's that's what is generated, i would recommend adding this feature to your build as well.

Grommish · May 11, 2021, 3:24am

Yes. Looks like it.

CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_SERIAL_8250_DMA=y
CONFIG_SERIAL_CORE=y
CONFIG_SERIAL_CORE_CONSOLE=y
CONFIG_SERIAL_EARLYCON=y
CONFIG_SERIAL_OF_PLATFORM=y
CONFIG_USB_SERIAL_GENERIC=y
CONFIG_USB_SERIAL_KEYSPAN_MPR=y
CONFIG_USB_SERIAL_KEYSPAN_USA18X=y
CONFIG_USB_SERIAL_KEYSPAN_USA19=y
CONFIG_USB_SERIAL_KEYSPAN_USA19QI=y
CONFIG_USB_SERIAL_KEYSPAN_USA19QW=y
CONFIG_USB_SERIAL_KEYSPAN_USA19W=y
CONFIG_USB_SERIAL_KEYSPAN_USA28=y
CONFIG_USB_SERIAL_KEYSPAN_USA28X=y
CONFIG_USB_SERIAL_KEYSPAN_USA28XA=y
CONFIG_USB_SERIAL_KEYSPAN_USA28XB=y
CONFIG_USB_SERIAL_KEYSPAN_USA49W=y
CONFIG_USB_SERIAL_KEYSPAN_USA49WLC=y
CONFIG_USB_SERIAL_SAFE_PADDED=y

broly · May 11, 2021, 3:35am

have you tried with console=ttyS0, as opposed to ttys1?

just going off this here:

Grommish · May 11, 2021, 4:21am

I'm giggling a bit because that's actually my repo

It looks like openwrt requires the bootargs, but it wasn't present in the dts/dtsi files I'm using from the OEM kernel (I had to flatten them because they were overlays). I'm not using the OpenWrt mt7621.dtsi for example, I'm using the one from Ubiquiti.

So, I added

        chosen {
                bootargs = "console=ttyS0,57600";
        };

to my mt7621.dtsi and will test it out.

Grommish · May 11, 2021, 6:19am

Well..

I think I managed to build out a x86_64 er50 kernel and loaded it onto a mips_24kc device.. I can't imagine why it won't work..

When I originally setup the external kernel, I forgot (because I didn't know at the time) to include an ARCH=mips to my make menuconfig, so, it "defaulted" to the host arch, and then tried to apply the ubnt_er_e50_defconfig in a x86_64 environment ...

I'm trying to build out with it set to actually be mips.. and I'll report back.

Grommish · May 11, 2021, 8:55am

Didn't change.. Still getting the just stopping at Booting the kernel.

I've noticed a few things.. The .bin size is very small (4mb). That's the defined kernel size in my device defines.. It's like the squashfs isn't part of that initramfs bin..

The size of the file under my main branch 5.10 build is 15mb...

Ideas?

broly · May 11, 2021, 12:24pm

the squash shouldn't affect whether or not you get a boot log.

it does affect whether you get a full boot after the kernel initialisation, though.

i would check your DTS files again to be sure, and i would use the openwrt ones instead of making changes to the ones you have.

this does smell like an issue related to the DTS since your configuration looks correct. the mt7621 does not really have issues like this. i would also check and be sure your bootargs variable in u-boot. it may be that the ttyS1,57600n8 is coming from your bootloader and overriding the ttyS0,57600n8 in your kernel configuraiton.

Grommish · May 12, 2021, 12:59am

I'm going to ditch the 19.07 and go for my main branch build. I can get that build system to build a bootable image for OpenWrt's 5.10, so I'll try to build out the 4.14 for it and see what happens.

broly · May 12, 2021, 1:02am

from what i can see, most of the mt7621 stuff is now on 5.10.

from what i also saw, your config build file was spitting out (what seems to be) 5.10 build settings for a 4.14 kernel. this may be part of the problem.

just try a 5.10 build. it shouldn't hurt. maybe driver compatibility issues will arise for edgerouter stuff, but at least you'll see a boot log to reach that conclusion.

Grommish · May 12, 2021, 1:05am

I've got a 5.10 build and only partially works (Switch Setup for Edgerouter 10X (RTL8367RB))

The issue is that the 5.x drivers for the RTL8367RB switch AND the MT7621 (which is now DSA) aren't the proper ones that are needed to make the ER10x actually work properly. My ultimate goal is to rip out the drivers from the 4.14 kernel and try to get them into the ramips target.

broly · May 12, 2021, 1:08am

cool, that makes sense.

i think the issue really is the DTS files. i know i haven't been able to keep my mt721.dtsi up to date with the latest in the openwrt tree since they've made a few structural changes (DSA being one, as you noted) that prevent me from doing so.

they also added things like a 'gmac' node to the ethernet driver and such, which are not compatible with the old 4.14 driver that has hardware nat support.

these are just some things i have observed.

Grommish · May 12, 2021, 1:10am

nod I'm not actually using the dts/dtsi files from OpenWrt, but a flattened from overlay file that I make available to the build system. I was looking to use 19.07 because I know 4.14 would, in theory, be no issue. Either way, I keep both repos local, so will test until I find something I'll post my dts/dtsi files if you're interested in looking

broly · May 12, 2021, 1:12am

to be honest, if you can't get a serial output, i think it's either your cmdline arguments aren't being passed. but since you have a 5.10 build partially working that rules out the u-boot arguments of ttys1 being a problem.

since you added the console argument to the dts i can't see anythign else being an issue. it sucks we can't even see the first few lines of boot

Grommish · May 12, 2021, 1:17am

I see the same bootargs being passed from uboot in the 5.10, so OpenWrt is probably injecting them elsewhere We will see:)

Automatic boot of image at addr 0x80A00000 ...
## Booting image at 80a00000 ...
   Image Name:   MIPS OpenWrt Linux-5.10.26
   Image Type:   MIPS Linux Kernel Image (uncompressed)
   Data Size:    15022737 Bytes = 14.3 MB
   Load Address: 80001000
   Entry Point:  80001000
   Verifying Checksum ... OK
OK
No initrd
## Transferring control to Linux (at address 80001000) ...
>>>>> mips_linux.c, 206: commandline = console=ttyS1,57600n8 ubi.mtd=7 root=ubi0_0 rootfstype=ubifs rootsqimg=squashfs.img rootsqwdir=w rw model=e55 ubnt_ramsize=512
## Giving linux memsize in MB, 256

Starting kernel ...



OpenWrt kernel loader for MIPS based SoC
Copyright (C) 2011 Gabor Juhos <juhosg@openwrt.org>
Decompressing kernel... done!
Starting kernel at 80001000...

broly · May 12, 2021, 1:20am

really appreciate your persistence.

it's what you need to make a name for yourself in this game.

you're already doing it. good job

broly · May 22, 2021, 1:58am

@grommish are you okay bud? hopefully you didn't give up.

maybe you can find a github with an older openwrt tree. that should work right?