Cisco Meraki MX64/MX65 image support

completely different (not repeating), but always with ..."do not mark it as bad".

Is anybody getting these messages? I checked two mx64 both show same, about few a day.

Would installing Debian be possible by any chance, once Openwrt is installed? Specs seems to be appropriate for it.

That's looks a lot to me, the flash chips are probably worn. Were the units laying for a long between OpenWrt installation and the moment you started using them?

For testing, you might want to temporarily fill the free space on filesystem, so stale PEBs get rewritten and weak bits disappear, and monitor the situation afterwards.

You can always have it in chroot. Getting it running natively is defnitely possible, but will require substantial effort.

1 Like

Theoretically, yes - realistically, not at all.

On one of the two mx64 during boot in dmesg, where information about ubi is, it says that 4 are bad.

A bit poetic :slight_smile:
Any more detail?

If they are the last four blocks of the chip, then this is bad block table, which itself is intentionally marked bad.

1 Like

You'd need to solve:

  • bootloader support (mainline u-boot, so Debian's u-boot inherits device support)
  • flash-kernel support (Debian's orchestration to install/ update u-boot)
  • full mainline kernel support, so Debian's kernel inherits support
    • include the necessary config changes into their multi-platform kernel
  • teach initramfs-tools about NAND and ubi
  • (d-i support)

Additional hurdles:

  • 1 GB non-replacable NAND flash might be huge for OpenWrt, but for Debian that is very tight
  • 2 GB RAM is o.k., not great
  • ARMv7/ armhf's days with Debian are numbered, there are already calls to drop it
  • non-replacable NAND and log churn is a sure way to kill the internal flash quickly
  • there is no boot fallback, no booting from removable media, if you brick it, the device is gone

The details are all yours, you want to make it happen.

Is it theoretically possible, yes.
Is that realistic, no.
Will others do it for you, muahaha, dream on...

Debian support needs:

  • full mainline support (kernel)
  • mainline u-boot or (even better) a u-boot+tianocore+grub-efi boot chain (ARM system ready)
  • removable media (SSD, HDD)
  • 'enough performance' (CPU, RAM, storage) to remain fun long term
  • idiot-safe recovery means (being able to boot from removable media)
1 Like

The reason I highlighted it is that on another mx64 on the same line in dmesg during boot is says 0.

Very detailed, highly appreciated, quite sure others will find it handy for thoughts about Openwrt to Debian transformations on (semi-)capable hardware :sun_with_face:

The main issue here, which basically prevents everything else from being reasonable, is the flash situation. You really want 'good' main storage (SSD or at least sdhc/ USB stick, which are at least cheap & easy to replace when they fail) - and the SOC needs to be able to boot from this storage reliably (bcm53xx can't).

Development gets really hard, if you can't plug in a USB stick or sdhc card to boot from/ recover, because you will 'brick' your installation (frequently) during the early development. And raw NAND flash will die quickly with an ordinary desktop linux distribution (excessive flash writes from logs, journals, etc. - something OpenWrt avoids at all costs, but which is necessary for a normal linux and the software running on it), as you can see from the RPi crowd[*]. This means you either need 'good storage', as in M.2 or SATA SSD, or 'cheap' storage you can replace easily (sdhc card, USB stick) - either of these would also elegantly solve the size issue (1 GB is really, really tight for Debian).

Your typical ARM SBC (RPi, sunxi, rockchip, etc.) can just boot from removable sdhc cards, USB or SSD, meaning you can't really brick it - plug in a new -known-good- sdhc card and off you go, so you can wildly experiment without risking to break it. Still hard development work, but no risk.

Your typical plastic router can't, if you break the internal flash or clobber its bootloader partition (or wifi calibration, where applicable), it's dead for good.

--
[*] in the RPi ecosystem, before the RPi4/ RPi5 which brought SSD support, it was commonly known to see sdhc cards die within 0.5-1.5 years of normal usage. And there we're talking about 32+ GB sdhc cards, with internal wear-leveling, so a lot more spare sectors to remap broken ones.

1 Like

Highly appreciated :+1:

I looked up type of NAND installed, it seams to be SLC, which is as I know among most reliable/endurant types.
Not sure which is the controller (probably CPU itself) to check which wear leveling tools exist

you may want to wander over to archlinuxarm to see what's possible.

anyway, flashed mx65 to openwrt, do not see the same issues you have.

1 Like

So far applies to two mx64, not present on mx65

Hi,

I tried to set-up an MX65 with openwrt. I did:

###########################################################################

U-boot installation: MX65 Only

Prepare a USB drive formatted to FAT. Download uboot_mx65 to the USB drive from the following location:
https://github.com/clayface/U-boot-MX64-20190430_MX65

Once you have telnet access to the MX65, plug in the USB stick and run the following commands. This is the most dangerous step so please ensure the dd command is entered correctly:
cd /tmp/media/sda1
dd if=uboot_mx65 of=/dev/mtdblock0

It seems to have worked fine. However, now I can't get the router to boot on openWRT.
I have openwrt-bcm5862x-generic-meraki_mx65-initramfs-kernel.bin on the / of the USB Drive (which I have now formater to FAT16 (it's a 4GB disk, so I made a new FAT16 partition of 32MB).

When I plugin the power with the reset button pressed, I see the orange light on and then the orange and white light alternate for a few sec (so I release the reset button). The light stays orange and when I plug an ethernet to any of the LAN, I can see that the RX light turn on but NOT the TX light. So when I try to ping the router, I get nothing.

Is there anything else I can do? Does that mean I bricked the router when I did: dd if=uboot_mx65 of=/dev/mtdblock0 ?

EDIT: Nevermind, the issue was that the right file to use is openwrt-bcm53xx-generic-meraki_mx65-initramfs.bin

1 Like

Also wondering.
Did you succeed to build it with POE?

Unfortunately not yet. I didn't have much time.
I think this [ https://github.com/mbilker/openwrt/commit/481c17d81b8b22f25519610ad8c9bd99dfa1c4cc ] can be a good starting point, since it's already based on kernel 6.6.
Please let me know if you try

1 Like

You can rebase this branch on main and build. POE is working.

I cherry-picked the commit onto 24.10.2 and now PSE is working fine, also it's really nice to have access via /sys/class/poe/.
This is how I did it (disclaimer: I’m not a git expert, so this might not be the best way to do it, but it worked for me):

git clone https://github.com/openwrt/openwrt.git -b v24.10.2 && cd openwrt
git remote add -f mx65_custom https://github.com/mbilker/openwrt
git cherry-pick 38be5c1fca574bf7f3a09bde08b59a4cbd8adfd8

Then I just followed the usual steps to configure and build. this guide was super helpful: https://gist.github.com/chankruze/dee8c2ba31c338a60026e14e3383f981

4 Likes