Hi,
I have an extra Banana Pi BPI-R3 Mini that I am happy to ship to someone to help promote the development of an Openwrt-supported release. The blob from
https://wiki.banana-pi.org/Banana_Pi_BPI-R3_Mini
I haven't had any luck using the SINOVOIP supplied version (I'm just using the version shipped on the board). https://github.com/BPI-SINOVOIP/BPI-R3MINI-OPENWRT-V21.02.3
Thanks,
Tim
I am just curious about this, the normal version BPI-R3 image can't boot the mini?
Hi
I saw from the banana pi forum, https://forum.banana-pi.org/t/bpi-r3-mini-emmc/16765/3, that mainline OpenWrt soon will support the BPI-R3 mini. I hope that´s true.
@daniel, do you have any news?
@daniel - Can you update here. I would also like to know. In theory should be able to take the Source for BPI-R3 and modify considering similar schematic, but don't really want to do so if you are already close.
Both ethernet ports on the Mini use - AirOha EN11H Phy https://www.airoha.com/products/p/tKkm7DPXi5m6wY2D. Interestingly enough Bananna pi which claims to be open source has excluded this page from the released schematic.
There does not appear to be a driver for this in Openwrt. There is a driver for it in ImmortalWRT. I am trying to use the ImmortalWRT driver and their pinout from the dts file for the mini to make this work. Getting closer but out of time for the day.
Very pretty 
After beating my head against a wall for some time I finally got this to compile and load. However in doing so, I also found I wasted my time as someone already has a patch. The issue... The Phy is not supported by mainline linux.
I recommend you use Mainline immortalwrt (not that old build that bpi shipps) until the Phy gets mainlined.
It seems there is no new thread about Official OpenWRT for BPI-R3 mini, so I think we can continue here. So we know that this device is compatible since this commit and I tried myself some new builds on it. I intended to let the original Sinovoip software on nand and use official OpenWRT on emmc, but I noticed that as soon as I install a software update to the emmc installation (using Luci) the nand is corrupted and the device can no longer boot from nand. Maybe this is caused by the different partition schemes. Also the nand recovery is very difficult because the new U-Boot that comes with OpenWRT cannot do much on the nand (I tried both mtd and nand commands but nothing happened) so I used the method from the banana pi forum - many thanks to frank-w! - to load his bl2 and fip by COM and then painfully load by ymodem the official mtk-bpi-r3mini-NAND-20231115-single-image.bin (1.5 hours...) and write it with mtd commands (I don't know, are there other methods in this u-boot build like ethernet-tftp or USB methods...?). Please @daniel, can you confirm these issues? Also during my first tests with both the original build and the official OpenWRT build I noticed a limited upload speed, under 200Mbps by wifi (and under 500 Mbps by Ethernet but this is not so concludent as I have only a USB adapter). Using wed or hw nat does not have big influence as the cpu is using less than 20% without any acceleration to do 900 Mbps nat download and less than 5% with wed.
LE: I was also able to recover much faster by using usb commands after loading the bl2 and fip by COM.
Yes, keeping SinoVoip image on NAND while running OpenWrt from eMMC is not supported. You can do it the other way around though (OpenWrt on SPI-NAND, SinoVoip stock on eMMC) and shouldn't experience any problems.
That's odd, accessing NAND in OpenWrt-built U-Boot on the R3 mini works fine for me, as does Ethernet support in U-Boot if you have the Airoha EN8811H firmware loaded on the flash or eMMC.
But is the new U-Boot aware of the different mtd partitions between SinoVoip/OpenWRT? Because using OpenWRT emmc U-Boot I tried to load the SinoVoip single image by tftp which worked without issues but writing using mtd or nand gave me errors or did nothing.
Meanwhile I was able to reproduce the slow upload problem which is not related to the wifi interface. I tested with iperf using only ethernet connections (1Gb adapters) and while the download (from wan to lan) is constant in the 850 Mbps range the upload is very variable:
[ 4] 15.01-16.00 sec 60.5 MBytes 513 Mbits/sec
[ 4] 16.00-17.01 sec 16.1 MBytes 134 Mbits/sec
[ 4] 17.01-18.01 sec 15.1 MBytes 127 Mbits/sec
[ 4] 18.01-19.01 sec 15.0 MBytes 126 Mbits/sec
[ 4] 19.01-20.00 sec 14.4 MBytes 122 Mbits/sec
[ 4] 20.00-21.01 sec 14.0 MBytes 117 Mbits/sec
[ 4] 21.01-22.00 sec 54.0 MBytes 454 Mbits/sec
[ 4] 22.00-23.01 sec 17.1 MBytes 143 Mbits/sec
[ 4] 23.01-24.00 sec 14.6 MBytes 124 Mbits/sec
[ 4] 24.00-25.01 sec 59.5 MBytes 497 Mbits/sec
[ 4] 25.01-26.00 sec 47.6 MBytes 402 Mbits/sec
[ 4] 26.00-27.02 sec 17.2 MBytes 143 Mbits/sec
[ 4] 27.02-28.00 sec 15.5 MBytes 132 Mbits/sec
[ 4] 28.00-29.00 sec 25.0 MBytes 210 Mbits/sec
[ 4] 29.00-30.01 sec 18.8 MBytes 155 Mbits/sec
Can anyone reproduce? I also noticed that the port does not work with some adapters, I've had problems with a laptop with Intel i219LM adapter, while Realtek adapters work fine.
I have not seen this kind of problem with my link partners (Marvell Tigon, Intel i200, various RealTek NICs, PHYs and switches; various MediaTek MT753x switches).
I assume the problem is the internal firmware of the Airoha EN8811H PHYs and it will be up to Airoha to resolve such issues as we don't have any sourcecode or other way to help ourselves with that.
Hence I'd like to ask you to help writing up a more detailed report including all details about the NICs on the other end of the link(s), use of RX/TX flow-control and whether the problem persists if you switch that on or off. That will allow us to forward the report to Airoha and hope they will be able to reproduce the problem and fix it.
If you want to use SinoVoip's image on eMMC you will also have to use their bootloader. I haven't tried to use OpenWrt's loader to work with SinoVoip image, it could work, but better use their loader as well then.
Accessing NAND from eMMC loader works fine for me, but of course, it will assume OpenWrt's NAND layout:
F0: 102B 0000
FA: 1040 0000
FA: 1040 0000 [0200]
F9: 103F 0000
F3: 1006 0033 [0200]
F3: 4001 00E0 [0200]
F3: 0000 0000
V0: 0000 0000 [0001]
00: 0000 0000
BP: 2400 0041 [0000]
G0: 1190 0000
EC: 0000 0000 [2000]
T0: 0000 021A [010F]
Jump to BL
NOTICE: BL2: v2.9.0(release):OpenWrt v2023-10-13-0ea67d76-1 (mt7986-emmc-ddr4)
NOTICE: BL2: Built : 20:21:11, Feb 15 2024
NOTICE: WDT: Cold boot
NOTICE: WDT: disabled
NOTICE: CPU: MT7986 (2000MHz)
NOTICE: EMI: Using DDR4 settings
NOTICE: EMI: Detected DRAM size: 2048MB
NOTICE: EMI: complex R/W mem test passed
NOTICE: BL2: Booting BL31
NOTICE: BL31: v2.9.0(release):OpenWrt v2023-10-13-0ea67d76-1 (mt7986-emmc-ddr4)
NOTICE: BL31: Built : 20:21:11, Feb 15 2024
U-Boot 2024.01-OpenWrt-r0+25207-b8978b06d1 (Feb 16 2024 - 05:48:30 +0000)
CPU: MediaTek MT7986
Model: Bananapi BPi-R3 Mini
DRAM: 2 GiB
Core: 53 devices, 23 uclasses, devicetree: embed
spi-nand: spi_nand spi_nand@0: Winbond SPI NAND was found.
spi-nand: spi_nand spi_nand@0: 128 MiB, block size: 128 KiB, page size: 2048, OOB size: 64
MMC: mmc@11230000: 0
Loading Environment from MMC... OK
In: serial@11002000
Out: serial@11002000
Err: serial@11002000
reset button found
Loading Environment from MMC... OK
Net: eth0: ethernet@15100000
Cannot read EFI system partition
Cannot read EFI system partition
Failed to persist EFI variables
( ( ( OpenWrt ) ) ) [eMMC] U-Boot 2024.01-OpenWrt-r0+25207-b8978b06d1 (Feb 16 2024 - 05:48:30 +0000)
Press UP/DOWN to move, ENTER to select, ESC to quit
1. Run default boot command.
2. Boot system via TFTP.
3. Boot production system from eMMC.
4. Boot recovery system from eMMC.
5. Load production system via TFTP then write to eMMC.
6. Load recovery system via TFTP then write to eMMC.
7. Load Airoha EN8811H firmware via TFTP then write to eMMC.
8. Load BL31+U-Boot FIP via TFTP then write to eMMC.
9. Load BL2 preloader via TFTP then write to eMMC.
a. Reboot.
b. Reset all settings to factory defaults.
0. U-Boot console
Hit any key to stop autoboot: 3
Hit any key to stop autoboot: 2
MT7986> mtd list
List of MTD devices:
* spi-nand0
- device: spi_nand@0
- parent: spi@1100a000
- driver: spi_nand
- path: /spi@1100a000/spi_nand@0
- type: NAND flash
- block size: 0x20000 bytes
- min I/O: 0x800 bytes
- OOB size: 64 bytes
- OOB available: 24 bytes
- 0x000000000000-0x000008000000 : "spi-nand0"
- 0x000000000000-0x000000200000 : "bl2"
- 0x000000200000-0x000008000000 : "ubi"
MT7986> nand list
List of NAND devices:
* spi-nand0
- device: spi_nand@0
- parent: spi@1100a000
- driver: spi_nand
- type: NAND flash
- block size: 0x20000 bytes
- page size: 0x800 bytes
- OOB size: 64 bytes
- OOB available: 24 bytes
- 0x000000000000-0x000008000000 : "spi-nand0"
- 0x000000000000-0x000000200000 : "bl2"
- 0x000000200000-0x000008000000 : "ubi"
MT7986>
I decided to follow your advice and install OpenWRT on NAND using your instruction from the original commit but I stopped at the last step because I think there is an error at instructions to create the last "production" partition (if this is the real name). Can you give me the correct commands after writing the recovery with ubiupdatevol /dev/ubi0_4 /tmp/openwrt-*-bananapi_bpi-r3-mini-initramfs-recovery.itb ?
The correct name of the production volume is fit. Sorry for the mistake in the commit message, I probably caused quite some confusion there and didn't notice it until now.
root@OpenWrt:/# ubinfo -a
UBI version: 1
Count of UBI devices: 1
UBI control device major/minor: 10:127
Present UBI devices: ubi0
ubi0
Volumes count: 7
Logical eraseblock size: 126976 bytes, 124.0 KiB
Total amount of logical eraseblocks: 1008 (127991808 bytes, 122.0 MiB)
Amount of available logical eraseblocks: 0 (0 bytes)
Maximum count of volumes 128
Count of bad physical eraseblocks: 0
Count of reserved physical eraseblocks: 20
Current maximum erase counter value: 4
Minimum input/output unit size: 2048 bytes
Character device major/minor: 250:0
Present volumes: 0, 1, 2, 3, 4, 5, 6
Volume ID: 0 (on ubi0)
Type: dynamic
Alignment: 1
Size: 9 LEBs (1142784 bytes, 1.0 MiB)
State: OK
Name: ubootenv
Character device major/minor: 250:1
-----------------------------------
Volume ID: 1 (on ubi0)
Type: dynamic
Alignment: 1
Size: 9 LEBs (1142784 bytes, 1.0 MiB)
State: OK
Name: ubootenv2
Character device major/minor: 250:2
-----------------------------------
Volume ID: 2 (on ubi0)
Type: static
Alignment: 1
Size: 9 LEBs (1142784 bytes, 1.0 MiB)
Data bytes: 1113777 bytes (1.0 MiB)
State: OK
Name: fip
Character device major/minor: 250:3
-----------------------------------
Volume ID: 3 (on ubi0)
Type: static
Alignment: 1
Size: 2 LEBs (253952 bytes, 248.0 KiB)
Data bytes: 147456 bytes (144.0 KiB)
State: OK
Name: en8811h-fw
Character device major/minor: 250:4
-----------------------------------
Volume ID: 4 (on ubi0)
Type: dynamic
Alignment: 1
Size: 59 LEBs (7491584 bytes, 7.1 MiB)
State: OK
Name: recovery
Character device major/minor: 250:5
-----------------------------------
Volume ID: 5 (on ubi0)
Type: dynamic
Alignment: 1
Size: 88 LEBs (11173888 bytes, 10.6 MiB)
State: OK
Name: fit
Character device major/minor: 250:6
-----------------------------------
Volume ID: 6 (on ubi0)
Type: dynamic
Alignment: 1
Size: 806 LEBs (102342656 bytes, 97.6 MiB)
State: OK
Name: rootfs_data
Today I've made new tests with iperf3 installed locally on the BPI-R3 mini router and with my laptop connected on the other end of the cable. With both eth0 and eth1 the results were fine, in the 800-900Mbps range. I tested in both directions, having the laptop and the router server and client, in turn. So my opinion is there is no PHY firmware problem, but a nat or routing issue. I tried with irqbalance on and off, no difference.
Did you test uploading in the nat scenario, like having the iperf3 on the LAN side acting as a server and an iperf3 client on the WAN side?
Another question, maybe related: I noticed that cat /sys/kernel/debug/ppe1/bind
will always give empty results (only ppe0 is used), so why is there a ppe1? What is its purpose?
LE: found some possibly interesting information here but it is beyond my level of understanding especially when auto-translated...
OK, thanks for clarifying, now I was able to install OpenWRT on NAND (and I verified that an update from Luci does not alter the emmc SinoVoip installation). Here are the correct writing operations, if someone needs them (after running insmod mtd-rw.ko i_want_a_brick=1):
mtd write /tmp/openwrt-*-bananapi_bpi-r3-mini-snand-preloader.bin /dev/mtd0 ubidetach -m 1 ubiformat /dev/mtd1 ubiattach -m 1 volsize=$(wc -c < /tmp/openwrt-*-bananapi_bpi-r3-mini-snand-bl31-uboot.fip) ubimkvol /dev/ubi0 -N fip -n 0 -s $volsize -t static ubiupdatevol /dev/ubi0_0 /tmp/openwrt-*-bananapi_bpi-r3-mini-snand-bl31-uboot.fip cd /lib/firmware/airoha cat EthMD32.dm.bin EthMD32.DSP.bin > /tmp/en8811h-fw.bin ubimkvol /dev/ubi0 -N en8811h-fw -n 1 -s 147456 -t static ubiupdatevol /dev/ubi0_1 /tmp/en8811h-fw.bin ubimkvol /dev/ubi0 -n 2 -N ubootenv -s 126976 ubimkvol /dev/ubi0 -n 3 -N ubootenv2 -s 126976 volsize=$(wc -c < /tmp/openwrt-*-bananapi_bpi-r3-mini-initramfs-recovery.itb) ubimkvol /dev/ubi0 -n 4 -N recovery -s $volsize ubiupdatevol /dev/ubi0_4 /tmp/openwrt-*-bananapi_bpi-r3-mini-initramfs-recovery.itb volsize=$(wc -c < /tmp/openwrt-*-bananapi_bpi-r3-mini-squashfs-sysupgrade.itb) ubimkvol /dev/ubi0 -n 5 -N fit -s $volsize ubiupdatevol /dev/ubi0_5 /tmp/openwrt-*-bananapi_bpi-r3-mini-squashfs-sysupgrade.itb ubimkvol /dev/ubi0 -n 6 -N rootfs_data -m
Hi
I´m trying to install OpenWRT to the NAND, but I get an error on the second command, ubidetach -m 1.
I´m starting from eMMC, SinoVoip stock fw.
Someone who knows why?
root@OpenWrt:~# mtd write /tmp/openwrt-*-bananapi_bpi-r3-mini-snand-preloader.bin /dev/mtd0
Unlocking /dev/mtd0 ...
Writing from /tmp/openwrt-mediatek-filogic-bananapi_bpi-r3-mini-snand-preloader.bin to /dev/mtd0 ...
root@OpenWrt:~# ubidetach -m 1
ubidetach: error!: cannot detach mtd1
error 19 (No such device)
root@OpenWrt:~#
The instructions from the commit post are clear:
Installation instructions for NAND
0. Set boot switch to boot from eMMC (assuming OpenWrt is installed there
by instructions above. Using stock rom or immortalwrt does NOT work!)
So you need to install first OpenWRT on eMMC before you can put it also on NAND. Another option is to boot by mtk_uartboot using frank-w's bl2 and OpenWRT eMMC fip then write with u-boot commands all mtd + ubi partitions using an USB stick as I don't think you can use tftp (you don't have access to the eth firmware in this case).
Ok, I installed OpenWRT to eMMC, according the instructions, and OpenWRT starts from eMMC. 
But, when I try to install to NAND I get error, Could not open mtd device: /dev/mtd0
Any clue why?
BusyBox v1.36.1 (2024-03-26 09:37:37 UTC) built-in shell (ash)
_______ ________ __
| |.-----.-----.-----.| | | |.----.| |_
| - || _ | -__| || | | || _|| _|
|_______|| __|_____|__|__||________||__| |____|
|__| W I R E L E S S F R E E D O M
-----------------------------------------------------
OpenWrt SNAPSHOT, r25666-80b2288ea3
-----------------------------------------------------
=== WARNING! =====================================
There is no root password defined on this device!
Use the "passwd" command to set up a new password
in order to prevent unauthorized SSH logins.
--------------------------------------------------
root@OpenWrt:~# ls /tmp
TZ log resolv.conf.d
dhcp.leases openwrt-r3-mini-initramfs-recovery.itb run
dnsmasq.d openwrt-r3-mini-snand-bl31-uboot.fip shm
etc openwrt-r3-mini-snand-preloader.bin state
hosts openwrt-r3-mini-squashfs-sysupgrade.itb sysinfo
lib overlay tmp
lock resolv.conf
root@OpenWrt:~# mtd write /tmp/openwrt-r3-mini-snand-preloader.bin /dev/mtd0
Could not open mtd device: /dev/mtd0
Can't open device for writing!
root@OpenWrt:~#
You need to install kmod-mtd-rw then execute insmod mtd-rw.ko i_want_a_brick=1 in order for mtd devices to be writable.