1 (edited by samot 2006-02-06 14:23:08)

Topic: devices with 2 MB flash

ipkg update fails with
"mkdir: Cannot create directory `//jffs/usr/': Read-only file system"
Please help HEEELP!

Don’t worry, just kidding. However such massages confirms the demand of running OpenWrt in 2 MB flash devices. I know that wireless freedom is VERY VERY limited is this case.

I’d like to create eventually a "2 MB flash howto". So please send your comments. (And be sure that I know how
to generate system with writeable jffs – I just want to make it affordable for others and make such system usable)


Where we are:

2048 kB whole flash
-256 kB bootloader PMON or CFE
-1408 kB lzma loader, kernel, squashfs
(equals to 1372 kB of whiterussian/rc4/micro/openwrt-wap54g-squashfs.trx rounded up to nearest 64 kB eraseblock)
  -64 kB NVRAM variables
=======
  320 kB … 5 eraseblocks (64 kB each) for read/write filesystem.

Minimal count of free eraseblocks in jffs2 is 5. It means that jffs2 fits there, but is absolutely useless, as it do not allow write until it has more than 5 free eraseblocks. Moreover jffs2 in linux 2.4 seems not collecting garbage with fs size 6 eraseblocks.
I believe authors of jffs2 that minimal count of free eraseblocks is limited at 5 for good reason when fs size >> 5.

Update:
pre RC5 (people/nbd/whiterussian/micro/openwrt-brcm-2.4-squashfs.trx) is only 1260kB so it leaves 128kB free in jffs2. Cool!

What we can do:

1)    Save more on kernel. Hmm, hardly.

2)    Save more on squashfs.
Somebody can live without IPv6 support in busybox (and ping6).
Some more busybox applets can be left out (httpd, tee, bzip2, uniq, vi features, cron, telnet, uptime).
Where appropriate (WAP54G, Asus WL500g) you can throw away vlan stuff (vconfig and kernel support)
Using ImageBuilder or build root you can resign on installing packages and save ipkg.
Having generated keys one can put dropbear on the diet and use server only.
If you are really brave unselect telnetd (give dropbear keys and root passwd to squashfs)
But wait – I want to add my favorite applications…

3)    Modify jffs2
Look at linux/fs/jffs2/nodelist.h

#define JFFS2_RESERVED_BLOCKS_BASE 3
/* Number of free blocks there must be before we... */
#define JFFS2_RESERVED_BLOCKS_WRITE (JFFS2_RESERVED_BLOCKS_BASE + 2)
/* ... allow a normal filesystem write */
#define JFFS2_RESERVED_BLOCKS_DELETION (JFFS2_RESERVED_BLOCKS_BASE + 1)
/* ... allow a normal filesystem deletion */
#define JFFS2_RESERVED_BLOCKS_GCTRIGGER (JFFS2_RESERVED_BLOCKS_BASE + 3)
/* ... wake up the GC thread */
#define JFFS2_RESERVED_BLOCKS_GCBAD (JFFS2_RESERVED_BLOCKS_BASE + 1)
/* ... pick a block from the bad_list to GC */
#define JFFS2_RESERVED_BLOCKS_GCMERGE (JFFS2_RESERVED_BLOCKS_BASE)
/* ... merge pages when garbage collecting */

I’m afraid that it is not as easy as lowering those number. It is matter of jffs2 design. Well, let’s say that we could save 2 blocks here – not so bad – with the risk of filesystem crash.

Slightly better is jffs2 in kernel 2.6 - it needs 5 eraseblocks for mount (can be safely lowered to 4) and at least 3 free eraseblocks to allow writes (easily can be lowered to 2 with some risk of fs crash). Unfortunatelly kernel and open source bcm43xx wifi driver is longer (and wifi do not work on my device yet).


4)    Use jffs (jffs1) instead of jffs2. I tried. jffs has no compression and minimal eraseblock count 4. It also save 26 kB on packed kernel image (and some RAM as well). jffs can surprisingly work below 4 blocks – but without garbage collecting, so as it gets full, you have to erase whole fs.

5)    Any other flash fs?

6)    Use tmpfs as a root, populate it by gunzip (bzip2) tar, like Oleg did in the old hacked Asus WL500g firmware. Of course, keep root small, just for config. No ipkg installs. And do not forget manually sync before powerdown. Stop laughing. I personally like this approach: we save 36 kB of packed kernel image and 320 kB of free erase blocks. Multiply this by sqashfs compression ratio – lot of room. (for mbm: coordinate the work with https://dev.openwrt.org/ticket/85, Reduce time from reflash to fully booted system.)

7)    Buy a better device. The easiest way – nothing for us, of course.


Please somebody who is familiar with: is freifunk 2 MB image just carefully configured or is there something revolutionary?

What do you think?

Re: devices with 2 MB flash

You may find the answer here:

http://freifunk.net/wiki/FreifunkFirmwareEnglish

LZMA compression for kernel and squasfs: leaves more space for installable packages (ca. 1.4 Mb for the firmware which is smaller than the original OpenWRT)

Re: devices with 2 MB flash

We do have lzma'ed kernel and squashfs, that firmware has some of the base packages removed.

Re: devices with 2 MB flash

Sorry for my misinformation. Thought it was the reason.

Re: devices with 2 MB flash

I have a pile of WAP 54G v1.0 (20+) that i would love to get working with Openwrt.  The particular job I need them to do is Wireless client mode with routed natted connection on the one ethernet port.

If  I can help testing in any way I will

Mark Gibbons

Re: devices with 2 MB flash

I'm aproaching "production" state with openwrt-ized wap54g v1.0 (this message goes to my provider thru it)
I'll let you know. Thx for support.

Re: devices with 2 MB flash

Make it to include USB driver to support USB HUB + USB memory card readers. This way, I can use this on my U.S Robotics 5461 WiFi Router + Printer Server that only comes with a 2/8 MB flash/RAM.

Re: devices with 2 MB flash

pap2boy wrote:

Make it to include USB driver to support USB HUB + USB memory card readers. This way, I can use this on my U.S Robotics 5461 WiFi Router + Printer Server that only comes with a 2/8 MB flash/RAM.

Good point for devices which have USB port. Unfortunatelly most of devices with small flash have no USB.
Hw mod to support mmc card is also an option: I'm afraid for hobbyists only.
And is discutable whether it is worth adding flash if you have only 8 MB RAM.

Re: devices with 2 MB flash

samot wrote:

I'm aproaching "production" state with openwrt-ized wap54g v1.0 (this message goes to my provider thru it)
I'll let you know. Thx for support.

Oh this is great news!  Good work!

Let me test! Let me test!  I am used to the odd bricked Wap54g so dont worry about me breaking them.

KR's

Mark

Re: devices with 2 MB flash

There seems to be a project dedicated to routers with 2MB flash at: http://midge.vlad.org.ua/wiki/

Re: devices with 2 MB flash

sammot,

Freifunk has nothing special. Just "carefully configured" and some hacks. Some of that stuff is showing up in OpenWrt too (e.g. the do-not-use-libgcc-hack and the upgrade-jffs2-for-2.4-kernel-hack). The critical space is saved by a WEPless wl.o currently. Some things are different, e.g. I need iproute2 (full ip command) so I discarded "ifconfig" and "route" completely. You may check out my hacks with "cvs -d:ext:sven-ola@cvs.sf.net:/cvsroot/ff-firmware/ff" to take a deeper look.

HTH Sven-Ola

Re: devices with 2 MB flash

jochen wrote:

There seems to be a project dedicated to routers with 2MB flash at: http://midge.vlad.org.ua/wiki/

This is an OpenWRT-like mini-distribution for ADM5120 based routers. Can this be taylored for routers that use broadcom chips, i.e. U.S. Robotics 5461? Also, the USB part is still not supported. I am hoping to be able to use this to drive my U.S. Robotic 5461 router to run VoIPBlasters as shown on this post.

13 (edited by samot 2006-02-01 22:09:09)

Re: devices with 2 MB flash

sven-ola:
Thanks for info and pointing to your cvs. Interesting reading.
From quick look to patches it seems that you mean mainly adding bbc compression in upgrade-jffs2-for-2.4-kernel-hack.
Dynamic free space limits in jffs2 from kernel 2.6.15 should be also big help (128kB). I haven't  checked how difficult would be backport to 2.4. See http://www.linux-mtd.infradead.org/source.html#kernelversions
Let's hope that bcm43xx wifi will be reliable enough soon and we can use 2.6
It's clear that you can't use tmpfs root as you need keep rouiting info "nonvolatile"

jochen, openwrtvlad:
Yes, good work. Vlado had the idea to use tmpfs root before.
It is not so difficult to program that so I didn't need to reuse his code (but duplicating work is sad as usual)
I see that Vlado is discussing in thread http://forum.openwrt.org/viewtopic.php?id=4221
Maybe he'll contribute to kamikaze?

14 (edited by samot 2006-02-13 13:25:01)

Re: devices with 2 MB flash

Alternate boot scripts for tmpfs root on OpenWrt

Scripts detect if free flash is insufficient for jffs2 and then set up writeable root on tmpfs.
On devices with enough flash they work as original ones.
Scripts should be compatible with recent whiterussian (pre rc5) and kamikaze images.
I tested them on WAP54G (2MB flash/8MB RAM) and also ASUS WL-500G (4/16).

http://www.volny.cz/vanekt/openwrt/mount-tmpfs-root-0.2-whiterussian.tar.gz

For whiterussian: etc/preinit does not include new netmsg feature.
For kamikaze: do not use etc/init.d/S10boot - use original one.

Image up to 1728 kBytes long will work on 2 MB device.
You can work with openwrt as with usual squashfs image until power off or reboot.
If you want to save changes in files use command

   tarfs write

Content of root fs is written in tar.gz format into flash partition mtd4 (OpenWrt). If there is no other room, archive is saved into 32 kByte flash space under NVRAM partition (awfull hack). The compression of many small files is much better than in jffs2, you need only 5 kByte for symlinks.

On boot the saved archive is detected and unpacked.

Version 0.2:
To save RAM all directories but /etc are just linked to /rom by default (so they are read only).
Use
    mkrw file
or
    mkrw dir
to replicate directory structure in tmpfs and make file or directory writeable.
You can also remove file or directory from tmpfs and link original one from /rom
by command
    mkro file
    mkro dir

DRAWBACKS:

As all files are unpacked in RAM, keep tmpfs usage small. Especially on 8 MB RAM devices it is not wise to install anything by ipkg - generate new image by ImageBuilder instead (you can leave out ipkg package from list).
BTW: Do you know if tmpfs supports "inplace" executing?

Tar archive is not reset automatically after reflashing as jffs2 root. To remove all saved files

mtd erase OpenWrt

If 32 kB block under NVRAM is used you have

nvram commit

after erasing OpenWrt to keep NVRAM. See https://dev.openwrt.org/ticket/244

Scripts do not handle changes in image size - if you flash an image which differs in number of erase blocks, you loose tar archive.


TODO:
huge fat warning that tmpfs is used should be displayed at login
wiki doc
simplify scripts

Re: devices with 2 MB flash

Some ideas :

-256 kB bootloader PMON or CFE

Compress the bootloader : for my work, I did a first stage bootloader that decompress gziped uboot from flash. The first bootloader is about 20KB. You coulf get high gzip compression ratio with AdvanceCOMP. Moreover you could reuse the decompress code from the first stage bootloader in the second stage bootloader (in my case for the kernel image).

-64 kB NVRAM variables

Can't the nvram variables could be putted on the jffs or in another place in order to don't eat a whole sector ?

16

Re: devices with 2 MB flash

256 kB bootloader PMON or CFE

That's just the allocated space, they really don't take the full space, they just expect the firmware to be at the 256k mark. Also of note - we have no intention of changing the bootloader since this isn't trivial for most people to restore after a bad flash or incompatible hardware.

Can't the nvram variables could be putted on the jffs or in another place in order to don't eat a whole sector ?

Not without rewriting the bootloader (the bootloader itself uses a few variables in nvram or creates nvram if it doesn't exist)

PS - many changes have already been made to reduce the size of rc5

Re: devices with 2 MB flash

Matc wrote:

Can't the nvram variables could be putted on the jffs or in another place in order to don't eat a whole sector ?

Kamikaze does not need nvram for config. If you changed the bootloader change also mtd mapping in kernel and you can go without nvram.

I agree with mbm that a bootloader change is not for everybody.

mbm wrote:

PS - many changes have already been made to reduce the size of rc5

Except the possibility run without jffs2.

Re: devices with 2 MB flash

On a device with both 2MB Flash and USB, couldn't you connect some type of USB drive and use that as the main drive?

19

Re: devices with 2 MB flash

What about using a network file system (NFS, SMBFS) for parts of the firmware?

The network file system could be on a client (for those parts of the firmware
which are only needed with a client) or on a host in the internet (connected via DSL).

If there is enough RAM, a tmpfs could be used which gets its files by an initial download
from some host using any network protocol (NFS, SMBFS, HTTP, FTP, ...).

I use a device with an NFS root file system. This solution is good for development
(nearly unlimited file space), but works also if you have a configuration with one
device with enough flash memory (or a server) which is always on and secondary
devices using NFS root.

Re: devices with 2 MB flash

Good work samot.  This rocks!

You said that the alternate boot script method should work on pre-rc5 whiterussian firmware.  Having become an unwilling expert on debricking my 54Gv6, now running rc5, I thought I would give it a try anyway.  As far as I can tell, it works!  I'm not running stock rc5, instead I am using the ImageBuilder to make my own custom micro firmware, which now includes your boot scripts.

The output of "tarfs write" looks a little strange though:

Compressing files...
flashing...
Bad trx header
If this is a firmware in bin format, like some of the
original firmware files are, use following command to convert to trx:
dd if=firmware.bin of=firmware.trx bs=32 skip=1
TRX check failed!
Unlocking OpenWrt ...
Writing from /tmp/rootfs.tar.gz to OpenWrt ...  [w]
done.

The error messages made me reach for my debricking hat, but the reboot went smoothly.  I haven't check the serial console boot output yet, but all external appearances are promising so far.

Again, good work.

Re: devices with 2 MB flash

driven wrote:

The output of "tarfs write" looks a little strange though:

Bad trx header
If this is a firmware in bin format, like some of the
original firmware files are, use following command to convert to trx:
dd if=firmware.bin of=firmware.trx bs=32 skip=1
TRX check failed!

Strange message comes from "mtd write" command and could be safely ignored.

Be careful as my scripts are obsolete as I haven't kept up them with recent openwrt development.
The basic idea (running without jffs2) is still valid but I haven't used a 2MB flash device for 6 months. Sorry.

Re: devices with 2 MB flash

I managed to get a Belkin v1444 (2MB Flash) to switch to a USB Flash drive for root.
I suppose the SD card mod could be used for routers without USB.

Re: devices with 2 MB flash

I spoke too soon.  The newly created tarfs survives a reboot, but not a power cycle.  The rc5 boot scripts erase the OpenWrt partition in preparation for putting the jffs on it.  I am looking at the scripts right now to figure out what to do about it, but it would help if I knew which release the samot scripts were based on.  I downloaded rc4 but given the number of changes I can't tell if that's the one.

Re: devices with 2 MB flash

I got it working.  The fix is simple.  In /etc/init.d/S99done change

firstboot switch2jffs

to

firstboot auto

There is probably a more correct way to do it, but this is what I have at the moment.  I don't know what this will do to a 4MB flash router.  Since I already have to use custom package and file lists in the ImageBuilder to build 2MB images it is easy to make these scripts 2MB specific.  Hopefully the 1.0 release, whatever it may be called, will have a similar capability built in.

Re: devices with 2 MB flash

My solution was to just use the script that switches the root filesystem to USB.
It should work with a SD card with the proper modifications, but I have not verified this. However, the 1 bit SD interface is very slow - 200kBps compared to 750kBps for USB 1.1. Use USB if it's available.