OpenWrt Forum Archive

Topic: 8.09.1 booting from USB stick

The content of this topic has been archived on 7 May 2018. There are no obvious gaps in this topic, but there may still be some posts missing at the end.

Hi,

Is there already somebody who managed to boot 8.09.1 from external device like USB stick?
I think right now it doesn't matter what kind of router you have, because I think this process should be similar by most of them.

My router is Asus WL-500W, and I would like to boot from the USB stick.
I've followed this HowTo (even though i know it was made for the previous 7.09 version, but I've adapted what I could, like replacing that 'ipkg' with 'opkg' , etc. ...):
    http://wiki.openwrt.org/oldwiki/openwrt … diahowto?s[]=500w

I've done everything as described, but the system won't start from the USB stick. Always booting from the router's flash drive.
This is how does it looks like after the boot:

# df -h
Filesystem                Size      Used Available Use% Mounted on
rootfs                    1.6M      1.6M         0 100% /
/dev/root                 1.6M      1.6M         0 100% /rom
tmpfs                    14.9M    400.0k     14.5M   3% /tmp
/dev/mtdblock/4           5.7M      2.8M      2.9M  49% /jffs
mini_fo:/jffs             1.6M      1.6M         0 100% /


# mount
rootfs on / type rootfs (rw)
/dev/root on /rom type squashfs (ro)
none on /dev type devfs (rw)
proc on /proc type proc (rw)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw)
/dev/mtdblock/4 on /jffs type jffs2 (rw)
mini_fo:/jffs on / type mini_fo (rw)
none on /proc/bus/usb type usbfs (rw)

Does everybody have clue what's wrong, why doesn't it boot from the USB stick and how to solve it?

thx in advance for any help


p.s. sorry for placing it to a wrong forum

Problem resolved.

After some examination of openwrt wiki I've found a way how to boot 8.09.1 on a router from external drive.

1.
Install necessary kernel modules for your external drive and FS (in my case USB stick and the ext3 FS)

opkg update
opkg install kmod-fs-ext2 kmod-fs-ext3 fdisk e2fsprogs kmod-usb2 kmod-usb-storage kmod-usb-ohci kmod-usb-uhci kmod-usb-uhci-iv

After installing the modules, you should either reboot the device or load the installed modules manually using the insmod command.

2.
Prepare the partition of your ext. device using the

fdisk /dev/scsi/host0/bus0/target0/lun0/part1

and format it

mkfs.ext3 /dev/scsi/host0/bus0/target0/lun0/part1

*** Replace the "/dev/scsi/host0/bus0/target0/lun0/part1" device with your own if differs ***

3.
Now copy everything from the flash to the USB device
mount it

mount -t ext3 /dev/scsi/host0/bus0/target0/lun0/part1 /mnt

*** Replace the "/dev/scsi/host0/bus0/target0/lun0/part1" device with your own if differs ***
now, we need to mount the root contents somewhere so we can copy them without copying /mnt and /proc to the new usb device

mkdir /tmp/root

if you're using squashfs, you want to copy the squashfs contents, not the jffs symlinks, so use the command:

mount -o bind /rom /tmp/root

now /tmp/root contains the root filesystem with no extra directories mounted so we just copy this to the usb device

cp /tmp/root/* /mnt -a

and now we unmount both the usb and our /tmp/root

umount /tmp/root
umount /mnt

4.
Next, remove the /sbin/init from the JFFS partition (this is just a symlink to BusyBox anyway):

rm /sbin/init

And replace it with this script:

#!/bin/sh
# change this to your boot partition
boot_dev="/dev/scsi/host0/bus0/target0/lun0/part1"
# *** Replace the "/dev/scsi/host0/bus0/target0/lun0/part1" device with your own if differs ***
# install needed modules for usb and the ext3 filesystem
# **NOTE** for usb2.0 replace "uhci" with "ehci-hcd"
# **NOTE** for ohci chipsets replace "uhci" with "usb-ohci"
for module in usbcore ehci-hcd scsi_mod sd_mod usb-storage jbd ext3; do {
        insmod $module
}; done
# this may need to be higher if your disk is slow to initialize
sleep 10s
# mount the usb stick
mount "$boot_dev" /mnt
# if everything looks ok, do the pivot root
[ -x /mnt/sbin/init ] && {
        mount -o move /proc /mnt/proc && \
        pivot_root /mnt /mnt/mnt && {
                mount -o move /mnt/dev /dev
                mount -o move /mnt/tmp /tmp
                mount -o move /mnt/jffs /jffs 2>&-
                mount -o move /mnt/sys /sys 2>&-
        }
}
# finally, run the real init (from USB hopefully).
exec /bin/busybox init

Make sure your new /sbin/init is executable:

chmod a+x /sbin/init

Now just reboot, and if you did everything right it should boot from the USB device automatically.

A better way would be to change the pre-init script to allow booting from both.

The following script checks if the USB drive is bootable. If its not it will boot from flash.

#!/bin/sh
# Copyright (C) 2006 OpenWrt.org
export PATH=/bin:/sbin:/usr/bin:/usr/sbin
. /etc/diag.sh

failsafe_ip() {
    ifconfig $ifname 192.168.1.1 netmask 255.255.255.0 broadcast 192.168.1.255 up
}

failsafe() {
    [ -n "$ifname" ] && grep "$ifname" /proc/net/dev >/dev/null && {
        failsafe_ip
        netmsg 192.168.1.255 "Entering Failsafe!"
        telnetd -l /bin/login.sh <> /dev/null 2>&1
    }
    lock /tmp/.failsafe
    ash --login
}

mount_usb() {
        boot_dev="/dev/sda2"
        mount_point="/mnt"

        echo "# Loading modules for usb"
        insmod usbcore
        insmod ehci-hcd
        insmod scsi_mod
        insmod sd_mod
        insmod jbd
        insmod ext2
        insmod ext3
        insmod usb-storage

        echo "# Waiting for devices to come up"
        sleep 10

        [ -e $boot_dev ] || {
          echo "$boot_dev NOT found"
          return 1
        }
        echo "# $boot_dev found"

        mount -o rw $boot_dev $mount_point || {
          echo "mount $boot_dev $mount_point failed"
          return 1
        }
        echo "# $boot_dev mounted on $mount_point"

        [ -e $mount_point/usb.boot ] || {
          echo "Missing file /usb.boot on media - NOT bootable, umounting."
          umount $mount_point
          return 1
        }
        echo "# Medium seems to be bootable"

        mount -o move /proc $mount_point/proc && \
          pivot_root $mount_point $mount_point/rom && {
            mount -o move /rom/dev /dev
            mount -o move /rom/sys /sys
            mount -o move /rom/tmp /tmp
          }
        echo "# Changed root to USB medium"

        return 0
}

mount proc /proc -t proc
mount sysfs /sys -t sysfs

size=$(awk '/MemTotal:/ {l=5242880;mt=($2*1024);print((s=mt/2)<l)&&(mt>l)?mt-l:s}' /proc/meminfo)
mount tmpfs /tmp -t tmpfs -o size=$size,nosuid,nodev,mode=1777

if grep devfs /proc/filesystems > /dev/null; then
    mount devfs /dev -t devfs
    M0=/dev/pty/m0
    M1=/dev/pty/m1
    HOTPLUG=/sbin/hotplug-call

elif [ -x /sbin/hotplug2 ]; then
    mount -t tmpfs tmpfs /dev -o size=512K
    mknod /dev/console c 5 1
    /sbin/hotplug2 --set-worker /lib/hotplug2/worker_fork.so --set-rules-file /etc/hotplug2-init.rules --no-persistent --set-coldplug-cmd /sbin/udevtrigger
    /sbin/hotplug2 --set-worker /lib/hotplug2/worker_fork.so --set-rules-file /etc/hotplug2-init.rules --persistent &
    M0=/dev/ptmx
    M1=/dev/ptmx
    HOTPLUG=

elif [ -x /sbin/udevd ]; then
    mount -n -t tmpfs -o mode=0755 udev /dev
    /sbin/udevd --daemon
    /sbin/udevtrigger
    /sbin/udevsettle
    M0=/dev/pty/ptmx
    M1=/dev/pty/ptmx
    HOTPLUG=
fi

mkdir -p /dev/pts /dev/shm
mount devpts /dev/pts -t devpts

# the shell really doesn't like having stdin/out closed
# that's why we use /dev/pty/m0 and m1 as replacement
# for /dev/console if there's no serial console available
dd if=/dev/console of=/dev/null bs=1 count=0 >/dev/null 2>/dev/null && {
    M0=/dev/console
    M1=/dev/console
}

exec <$M0 >$M1 2>&0

echo "- preinit -"
echo "Press CTRL-C for failsafe"
trap 'FAILSAFE=true' INT
trap 'FAILSAFE=true' USR1
[ -e /etc/preinit.arch ] && . /etc/preinit.arch
set_state preinit
echo "$HOTPLUG" > /proc/sys/kernel/hotplug
export FAILSAFE
eval ${FAILSAFE:+failsafe}
lock -w /tmp/.failsafe

if [ -z "$INITRAMFS" ]; then
    mount_usb || mount_root
    [ -f /sysupgrade.tgz ] && {
        echo "- config restore -"
        cd /
        mv sysupgrade.tgz /tmp
        tar xzf /tmp/sysupgrade.tgz
        rm -f /tmp/sysupgrade.tgz
        sync
    }

    echo "- init -"
    
    exec /sbin/init
fi

This way if anything happens to the usb drive you still have basic access to openwrt to fix the problem.

(Last edited by Mavy on 14 Aug 2009, 07:44)

I'm having an issue with this procedure on a Dlink DIR-320 with brcm-2.4-generic built with the required options to locate the flash in the required address and enabling USB.

#!/bin/sh
boot_dev="/dev/scsi/host0/bus0/target0/lun0/part1"
for module in usbcore ehci-hcd scsi_mod sd_mod usb-storage jbd ext2 ext3 ; do {
        insmod $module
}; done
sleep 4s

mount -t ext3 "$boot_dev" /mnt
 [ -x /mnt/sbin/init ] && {
        mount -o move /proc /mnt/proc && \
        pivot_root /mnt /mnt/mnt && {
                mount -o move /mnt/dev /dev
                mount -o move /mnt/tmp /tmp
                mount -o move /mnt/jffs2 /jffs2 2>&-
                mount -o move /mnt/sys /sys 2>&-
        }
 }

When executing this script I get the following message:
mount: /proc is not a block device

It would seem to me there's something wrong with the mount to move to /proc link across.

The USB key has been partitioned and formatted along with the files copied across as per the instructions. A directory of /mnt/proc exists on the USB key and is empty.

Can anyone shed any light onto what the problem might be?

(Last edited by Kartman on 16 Aug 2009, 12:53)

Mavy wrote:

A better way would be to change the pre-init script to allow booting from both.

The following script checks if the USB drive is bootable.

--snip--

This way if anything happens to the usb drive you still have basic access to openwrt to fix the problem.

Yes! But, I would go a few steps further:

1. I want to be able to *control* whether the router boots from USB or flash
2. I *don't* want to hardcode where the USB drive is (/dev/sda2 in your code that I snipped). The router should find it for me.
3. I never want to mount an ext2 or ext3 filesystem without at least preening (fsck -p) it

Finally, I want to know what the script did. So I wrote a souped-up pivotroot, outlined in these blog posts: http://www.kaufmanfamily.net/blog/2009/ … root-intro and http://www.kaufmanfamily.net/blog/2009/ … or-openwrt . If you skip the blog posts, that's fine, but note that I added a few requirements for packages to load (e2fsprog) and directories on both filesystems (/flash).

This goes in /etc/init.d/pivotroot:

#!/bin/sh
# From http://oldwiki.openwrt.org/UsbStorageHowto.html - DHK 7/14/09
#
# Greatly enhanced to select among (maybe) several USB Mass Storage devices
# Assumes that the root fs will be on an ext3 filesystem in partition 1,
# and looks for /flash and /jffs in the root directory. Also assumes that
# mounting and checking each candidate device is harmless. Runs e2fsck
# over partitions before mounting them; note that this causes spurious
# date-related complaints every time, since time has not been synchronized
# yet at the time pivotroot is run.
#
# Output from this script is stashed at /tmp/pivotroot.log.
#
# DHK 7/23/2009

# Set Power LED flashing while we look for OpenWRT root
if [ -f /etc/diag.sh ]; then
  LED=1
  . /etc/diag.sh
  POW=`cat /proc/diag/led/power`
  set_led power f
fi

# install needed modules for usb and the ext3 filesystem
# We could defer loading the filesystem modules until we know there's a useful partition.
# **NOTE** for usb2.0 replace "uhci" with "ehci_hcd"
# **NOTE** for ohci chipsets replace "uhci" with "usb-ohci"
# **NOTE** for WL-500gP usb-uhci not usb-ohci
echo -n "pivotroot loading kernel modules: "
for module in usbcore usb-uhci scsi_mod sd_mod usb-storage ext2 jbd ext3 ; do {
  echo -n "$module "
  insmod $module
}; done
echo
# this may need to be higher if your disk is slow to initialize
sleep 4s

PART1=`find /dev/scsi -name part1`

if [ -z $PART1 ]; then
  echo No partitions, skipping pivotroot
else
  # Look for a mountable USB stick with, maybe, OpenWRT on it
  for dev in $PART1; do
    echo pivotroot checking $dev
    e2fsck -p $dev
    fsck=$?
    echo fsck status is $fsck
    if [ $fsck -eq 2 ]; then
      echo Corrected errors on $dev, need to reboot
      sleep 5
      reboot
    elif [ $fsck -eq 1 ]; then
      echo Corrected errors on $dev
    elif [ $fsck -gt 2 ]; then
      echo No usable filesystem on $dev
      continue
    fi
    echo Mounting $dev
    mount -t ext3 $dev /flash
    mt=$?
    if [ $mt -eq 0 ]; then
      if [ -x /flash/sbin/init -a -d /flash/jffs -a -d /flash/flash ]; then
        echo Found OpenWRT root on $dev
        # Side-effect - leave /flash mounted
        break;
      else
        echo "Missing /sbin/init, /jffs, or /flash (mount status was $mt)"
        [ -x /flash/sbin/init ] || echo Failed -x /sbin/init
        [ -d /flash/flash ]     || echo Failed -x /flash
        [ -d /flash/jffs ]      || echo Failed -x /jffs
        umount $dev
      fi
    else
      echo mount status is $mt
    fi
  done

  # if everything looks ok, do the pivot root
  [ -x /flash/sbin/init ] && {
          mount -o move /proc /flash/proc && \
          pivot_root /flash /flash/flash && {
                  mount -o move /flash/dev /dev
                  mount -o move /flash/tmp /tmp
                  mount -o move /flash/jffs2 /jffs2 2>&-
                  mount -o move /flash/sys /sys 2>&-
          }
  }
fi

# Restore Power LED to previous state
[ $LED -eq 1 ] && set_led power $POW

This goes in /etc/init.d/rcS:

# Improved from http://oldwiki.openwrt.org/UsbStorageHowto.html
# Switch the root filesystem to USB, if present
# DHK 7/14/09
if [ $2 == "boot" -a -x /etc/init.d/pivotroot ] ; then
        /etc/init.d/pivotroot > /tmp/pivotroot.log
fi

(Last edited by dhkaufman on 10 Sep 2009, 16:08)

This how to does not work for me as it should be :-( ,  I tryed this how to on Linksys-nslu2 , .... but after some head scratching I have successed!
So I dont know if it is right or wrong but it just works fine, so here is what I have done:

I have followed this how to strictly but my init script is called /sbin/init.busb , and the original init script is left as it was by default i.e. unchanged!

#!/bin/sh
# change this to your boot partition
boot_dev="/dev/sda1"
# *** Replace the "/dev/scsi/host0/bus0/target0/lun0/part1" device with your ow$
# install needed modules for usb and the ext3 filesystem
# **NOTE** for usb2.0 replace "uhci" with "ehci-hcd"
# **NOTE** for ohci chipsets replace "uhci" with "usb-ohci"
for module in usbcore ehci-hcd uhci-hcd ohci-hcd scsi_mod usb-storage jbd ext3;$
        insmod $module
}; done
# this may need to be higher if your disk is slow to initialize
sleep 5s
# mount the usb stick
mount $boot_dev /mnt
# if everything looks ok, do the pivot root
[ -x /mnt/sbin/init ] && {
        mount -o move /proc /mnt/proc && \
        pivot_root /mnt /mnt/mnt && {
                mount -o move /mnt/dev /dev
                mount -o move /mnt/tmp /tmp
                mount -o move /mnt/jffs /jffs 2>&-
                mount -o move /mnt/sys /sys 2>&-
        }
}
# finally, run the real init (from USB hopefully).
exec /bin/busybox init

After that I modified /etc/init.d/done script , just added an execution line to init.bsub script, here is:

#!/bin/sh /etc/rc.common
# Copyright (C) 2006 OpenWrt.org

START=95
boot() {
        [ -d /tmp/root ] && {
                lock /tmp/.switch2jffs
                firstboot switch2jffs
                lock -u /tmp/.switch2jffs
        }

        # set leds to normal state
        . /etc/diag.sh
        set_state done
        ./sbin/init.busb
}

So finally after  reboot I have this:

root@OpenWrt:~# df
Filesystem           1k-blocks      Used Available Use% Mounted on
rootfs                  432260     19161    390781   5% /
/dev/root                 1472      1472         0 100% /mnt/rom
/dev/mtdblock5            4992      2576      2416  52% /jffs
mini_fo:/jffs             1472      1472         0 100% /mnt
/dev/sda1               432260     19161    390781   5% /mnt/mnt/usbdrive
/dev/sda1               432260     19161    390781   5% /

That is all, please let somebody linux-guru tell me wheter I can rely on this system ?!

Hi guys. Are any of you using the 2.6 kernel btw?

I am using the command from the Wiki

/sbin/hotplug2 --override --persistent --max-children 1 --no-coldplug &

but it gives me a missing worker name error.
Thank you
Misha

dhkaufman wrote:

Finally, I want to know what the script did. So I wrote a souped-up pivotroot, outlined in these blog posts: http://www.kaufmanfamily.net/blog/2009/ … root-intro and http://www.kaufmanfamily.net/blog/2009/ … or-openwrt . If you skip the blog posts, that's fine, but note that I added a few requirements for packages to load (e2fsprog) and directories on both filesystems (/flash).

When I tried to access the above blog links, the links spitted out this Error establishing a database connection. Can you please fix the links?

mazilo wrote:

When I tried to access the above blog links, the links spitted out this Error establishing a database connection. Can you please fix the links?

Sorry about that. All fixed now.

forest_gump wrote:

This how to does not work for me as it should be :-( ,  I tryed this how to on Linksys-nslu2 , .... but after some head scratching I have successed!
So I dont know if it is right or wrong but it just works fine, so here is what I have done:

I have followed this how to strictly but my init script is called /sbin/init.busb , and the original init script is left as it was by default i.e. unchanged!

...snip...

After that I modified /etc/init.d/done script , just added an execution line to init.bsub script, here is:

...snip...

So finally after  reboot I have this:

...snip...

That is all, please let somebody linux-guru tell me wheter I can rely on this system ?!

You can use the system, and it will be fine, but it will be confusing when you update your software or change your configuration.

What you have done is to make your system boot up entirely out of flash, then at the last second switch to the USB filesystem. So you have one set of system configuration, startup scripts, etc, on USB and another on flash - that is common to all these pivotroot setups. But the startup scripts that will get updated by the system tools - the ones on the USB filesystem - are not the ones that will run at system startup - system startup still runs off of flash. For example, if you add a new software package, and want it to start at system boot time, the startup script in /etc/init.d will be in the "wrong" /etc/init.d, and the software package won't start. By the time you pivot to the USB root, you've already run whatever startup scripts were in flash.

The other approaches do the pivot much sooner in the bootup process, so that fairly little code is run out of flash, and rather a lot is run off USB. That way the system is less confusing to maintain.

dhkaufman wrote:
mazilo wrote:

When I tried to access the above blog links, the links spitted out this Error establishing a database connection. Can you please fix the links?

Sorry about that. All fixed now.

Many thanks for your quick response. I guess I will give it a try with the better pivotroot (2nd link).

dhkaufman wrote:

Sorry about that. All fixed now.

On your blog, I am not sure if this is a bug or not; however, I noticed an inconsistency of using /jffs ([ -d /flash/jffs ] || echo Failed -x /jffs) and /jffs2 (mount -o move /flash/jffs2 /jffs2 2>&-) as shown below:

        echo "Missing /sbin/init, /jffs, or /flash (mount status was $mt)"
        [ -x /flash/sbin/init ] || echo Failed -x /sbin/init
        [ -d /flash/flash ]     || echo Failed -x /flash
        [ -d /flash/jffs ]      || echo Failed -x /jffs
        umount $dev
      fi
    else
      echo mount status is $mt
    fi
  done

  # if everything looks ok, do the pivot root
  [ -x /flash/sbin/init ] && {
          mount -o move /proc /flash/proc && \
          pivot_root /flash /flash/flash && {
                  mount -o move /flash/dev /dev
                  mount -o move /flash/tmp /tmp
                  mount -o move /flash/jffs2 /jffs2 2>&-
                  mount -o move /flash/sys /sys 2>&-
          }
  }
8472 wrote:

f you're using squashfs, you want to copy the squashfs contents, not the jffs symlinks, so use the command:

mount -o bind /rom /tmp/root

now /tmp/root contains the root filesystem with no extra directories mounted so we just copy this to the usb device

I don't see any partition(s) mounted under /rom directory. So, is the above step necessary and can't one just do a straight cp -a /rom /mnt?

mazilo wrote:

On your blog, I am not sure if this is a bug or not; however, I noticed an inconsistency of using /jffs ([ -d /flash/jffs ] || echo Failed -x /jffs) and /jffs2 (mount -o move /flash/jffs2 /jffs2 2>&-)

Yup, that looks like a bug. Based on the two systems I just looked at quickly, it should be jffs everywhere. I'm not sure where the jffs2 snuck in.

dhkaufman wrote:
mazilo wrote:

On your blog, I am not sure if this is a bug or not; however, I noticed an inconsistency of using /jffs ([ -d /flash/jffs ] || echo Failed -x /jffs) and /jffs2 (mount -o move /flash/jffs2 /jffs2 2>&-)

Yup, that looks like a bug. Based on the two systems I just looked at quickly, it should be jffs everywhere. I'm not sure where the jffs2 snuck in.

I hope you got your blog updated with bug fixes. Thanks.

dhkaufman wrote:
mazilo wrote:

On your blog, I am not sure if this is a bug or not; however, I noticed an inconsistency of using /jffs ([ -d /flash/jffs ] || echo Failed -x /jffs) and /jffs2 (mount -o move /flash/jffs2 /jffs2 2>&-)

Yup, that looks like a bug. Based on the two systems I just looked at quickly, it should be jffs everywhere. I'm not sure where the jffs2 snuck in.

I finally got a chance to look at this more closely, and I don't think it's a bug after all. There are two different things going on here: First, my pivotroot script uses the presence of a /jffs directory to decide whether a partition holds a valid OpenWRT image. Second, in the actual pivot, the filesystem mounted on the /jffs2 directory (if any) is moved to the new root.

In fact, my OpenWRTs don't have a filesystem mounted on /jffs2, but they do have a filesystem mounted on /jffs. That filesystem gets mounted by startup scripts that run after pivotroot, so it's correct that pivotroot doesn't pivot /jffs.

The code that moves /jffs2 comes from the OpenWRT wiki. I'm hesitant to change it, since it may be important on hardware that I don't have.

The only possible bug I see here is that maybe the presence of /jffs is a bad indicator of a valid OpenWRT system....

Thanks for your explanations and am sure others will appreciate that.

@dhkaufman--your blog post links are now dead (for me).  If you're around, can you update the links or post your investigations here?

lizby wrote:

@dhkaufman--your blog post links are now dead (for me).  If you're around, can you update the links or post your investigations here?

My blog got stomped by a Wordpress upgrade, but it's back now. Sorry for the outage.

@dhkaufman ... and any others

I have not succeeded in getting this to work on boot-up.  After copying files from /rom to the usb drive, if I run your code from the command line, modified to look only at "/dev/sda1", then it does the pivotroot successfully.  When invoked from rcS, pivotroot.log says "No partitions, skipping pivotroot".  I tried increasing the number of seconds, without success.

I tried forest_gump's method of doing the pivotroot as the last command in /etc/init.d/done.  I think it probably pivoted ok, but I was unable to login--it didn't like my password.  Removing the USB drive and rebooting got me back to a working system.

Perhaps in looking for "/dev/sda1" I am looking too early in the boot process.  I didn't understand the "/dev/scsi" in the find command in the script.  (Is that architecture-related? I'm doing this on a WL-520gU--brcm47xx.)  Perhaps it's related to a point in time in the boot process, but there is no /dev/scsi anything in my system after booting with a usb drive--but there is a "/dev/sda1".

Your instructions say to put the code which calls pivotroot into /etc/init.d/rcS before the "LOGGER=cat" line.  It didn't find a device (/dev/sda1).

I feel like I'm fairly close, but I'm not quite there.  Any suggestions would be appreciated.

lizby wrote:

Perhaps in looking for "/dev/sda1" I am looking too early in the boot process.  I didn't understand the "/dev/scsi" in the find command in the script.  (Is that architecture-related? I'm doing this on a WL-520gU--brcm47xx.)  Perhaps it's related to a point in time in the boot process, but there is no /dev/scsi anything in my system after booting with a usb drive--but there is a "/dev/sda1".

@lizby,

What OpenWRT release/kernel are you running? What kind of USB drive are you using? I am running Kamikaze 8.09 and 8.09.1 on an ASUS WL-520gU - actually I have done this trick on three of them. On my systems, a USB thumb drive always shows up as /dev/scsi/host0/bus0/target0/lun0 .

If I had your scenario, I would change the line in my pivtoroot script with the "find" command to be

PART1=/dev/sd?1

which should do the same job: list all the partition-1s on the system. Maybe that's what you already did.

Another possibility is that your drive isn't recognized until the system boots further, because you don't have the right modules loaded in pivotroot. Look at the output of lsmod, and compare it to the modules listed in the pivotroot script, and see if anything important-looking is missing. Sorry I can't be more specific.

I'm using trunk--tried both download of latest and self-built after svn up.  I've seen many mentions of scsi, but my NSLU2 systems have always had the hard drive come up as /dev/sd?1--to the best of my imperfect recollection.  I just started using the WL-520gU in December, and on that they have always come up the same.  I just today installed a download of trunk on a brand new WL-500gPv2, and got the same--usb recognized as /dev/sda1.  I've been using a Kingston 2gb usb drive.  It is recognized with a non-pivotroot boot, so I don't think there is a problem with that drive.  I'll try another one, tho.

I had not originally conned the obvious, that the insmod loop was there to load the modules which would cause the usb drive to be recognized, and if it was not recognized after that, then something was missing.  I've been banging on that today, but it hasn't fallen into place yet.  I trust I will get there by persuing that path.  Thanks for your suggestions.

I have not been able to get pivotroot to work on my WL-520gU at the point where dhkaufman put it (before "LOGGER=" in /etc/init.d/rcS).  With the serial cable, I can see that it is putting out messages to just the point before where it says "Please press enter to activate this console".  The boot process doesn't complete and the only way out is to reflash.

So I have tried to combine dhkaufman's code with the process used by forest_gump, putting the pivot root into /etc/init.d/done, after everything else has been done.  This appears to do the pivotroot, but with the caveats mentioned in the thread.

The packages installed are kmod-usb-core kmod-usb-storage kmod-usb2 kmod-fs-ext2 kmod-fs-ext3 kmod-usb-ohci e2fsprogs.  openssh-sftp-server is not required but is small and makes it easier to copy files to the router.

I have mounted the usb drive, formatted as ext3, on a directory named "usb".  Here is the code I used to format the drive and copy the internal flash to it (assuming that the drive is /dev/sda1 and partitioned as linux).

umount /dev/sda1
mkfs.ext3 /dev/sda1
mkdir -p /tmp/root
mount -o bind /rom /tmp/root
mount /dev/sda1 /usb
cp /tmp/root/* /usb -a
sync
echo "This is usb drive /dev/sda1 to be mounted on /usb" > /usb/sda1.dsk
umount /tmp/root
umount /usb

Here is df output with no usb drive plugged in:

Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/root                 1920      1920         0 100% /rom
tmpfs                     6784        32      6752   0% /tmp
tmpfs                      512         4       508   1% /dev
/dev/mtdblock3            1152       204       948  18% /jffs
mini_fo:/jffs             1920      1920         0 100% /

I put the call to pivotroot into /etc/init.d/done (and enabled it)

START=95
boot() {
        [ -d /tmp/root ] && {
                lock /tmp/.switch2jffs
                firstboot switch2jffs
                lock -u /tmp/.switch2jffs
        }

        # process user commands
        [ -f /etc/rc.local ] && {
                sh /etc/rc.local
        }
        # pivotroot to usb, if available -- added
        /etc/init.d/pivotroot

        # set leds to normal state
        . /etc/diag.sh
        set_state done
}

Here is my pivotroot code:

# using some of dhkaufman's code from
# https://forum.openwrt.org/viewtopic.php?id=21377
# look for usb drive as /dev/sda1; if found, unmount, e2fsck, if no error, boot
  DRIVE=`ls /dev/sda1`

  if [ -z $DRIVE ]; then
    echo No partitions, skipping pivotroot
  else
    umount $DRIVE
    e2fsck -p $DRIVE
    fsstat=$?
    echo e2fsck status is $fsstat
    if [ $fsstat -eq 2 ]; then
      echo Corrected errors on $DRIVE, need to reboot
      sleep 5
      reboot
    elif [ $fsstat -eq 1 ]; then
      echo Corrected errors on $DRIVE
    elif [ $fsstat -gt 2 ]; then
      echo No usable filesystem on $DRIVE
    fi
    echo Mounting $DRIVE
    mount -t ext3 $DRIVE /usb
    mt=$?
    if [ $mt -eq 0 ]; then
      if [ -x /usb/sbin/init -a -d /usb/jffs -a -d /usb/usb ]; then
        echo Found OpenWRT root on $DRIVE
   # Side-effect - leave /usb mounted
      else
        echo "Missing /sbin/init, /jffs, or /usb (mount status was $mt)"
        [ -x /usb/sbin/init ] || echo Failed -x /sbin/init
        [ -d /usb/usb ]     || echo Failed -x /usb
        [ -d /usb/jffs ]      || echo Failed -x /jffs
        umount $DRIVE
      fi
    else
      echo mount status is $mt
    fi
  fi
#if everything looks ok, do the pivot root
  [ -x /usb/sbin/init ] && {
     mount -o move /proc /usb/proc && \
     pivot_root /usb /usb/usb && {
       mount -o move /usb/dev /dev
       mount -o move /usb/tmp /tmp
       mount -o move /usb/jffs2 /jffs2 2>&-
       mount -o move /usb/sys /sys 2>&-
     }
   }

Here is the output from df when the device is booted with the usb drive in:

Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/root                 1920      1920         0 100% /usb/rom
tmpfs                     6784        32      6752   0% /tmp
tmpfs                      512         0       512   0% /dev
/dev/mtdblock3            1152       208       944  18% /usb/jffs
mini_fo:/jffs             1920      1920         0 100% /usb
/dev/sda1              1924720     42988   1783960   2% /

Is anything evidently wrong with this?

A couple of problems.  I would be stuck on this process as implemented without the serial cable.  When I tried to log in, I kept getting "Access denied".  Using the serial console, I ran "passwd" and set it to the same thing that I used for the device without the usb drive.  Then I could get in with the normal ssh client (putty, in my instance).  What causes this and how can it be avoided so that this process could be followed by someone without a serial cable?

Second problem: the date isn't correct.  ntpclient is running with the correct paramaters, but the date is wrong--based on Unix zero date.  How can this be fixed?

root@wl63:~# date
Wed Dec 31 19:10:30 EST 1969
root@wl63:~# ps | grep ntpclient
  754 root       832 S    /usr/sbin/ntpclient -i 600 -s -l -D -p 123 -h 0.openw

New packages are going to the usb drive.  I did "opkg update" and "opkg install nano", and could use nano.  After rebooting without the usb drive, I couldn't use nano, which is good because that indicates that it wasn't installed to the internal flash.

So, this method is close, but not perfect.  Any suggestions would be appreciated.

lizby wrote:

Any suggestions would be appreciated.

AFAIC, pivoroot should be called before run_scripts by /etc/init.d/rcS so that the system will boot off of the USB partition. If you call pivotroot at the end of the booting process, perhaps the /proc FS on your USB partition doesn't longer reflect to the boot information contained on the pivoted /rom/proc FS.

FYI, the /etc/init.d/rcS file on my WGT634U device which clearly indicates how pivoroot is called is shown below for your convenience:

#!/bin/sh
# Copyright (C) 2006 OpenWrt.org

run_scripts() {
      for i in /etc/rc.d/$1*; do
              [ -x $i ] && $i $2 2>&1
      done | $LOGGER
}

LOGGER="cat"
[ -x /usr/bin/logger ] && LOGGER="logger -s -p 6 -t sysinit"

#
# Switch to pivoroot if available
#
if [ "$2" == "boot" -a -x /etc/init.d/pivotroot ] ; then
      echo "[`date`]: Pivot root log created." > /tmp/pivotroot.log
      /etc/init.d/pivotroot >> /tmp/pivotroot.log
fi

if [ "$1" == "S" ]; then
      run_scripts "$1" "$2" &
else
      run_scripts "$1" "$2"
fi

@mazilo--thanks for pointing out that the pivotroot process I used might leave inconsistancies in the pivoted system.  As far as I understand it, except for issues related to smaller flash and ram, the WL-520gU should be able to run what the WGT634U does, so could you post your pivotroot code?

(Last edited by lizby on 29 Jan 2010, 20:23)

The discussion might have continued from here.