How to modify kernel parameters / command line after flashing?

I'm running LEDE 17.1.04 on the dualcore router O2 6431 and the default bootargs include nosmp because one core is occupied by the FXS part of the router.

But I want to use a few of these routers as VPN-gateways and/or for freifunk and both scenarios benefit greatly of a second CPU core...

Now LEDEs buildsystem does not jet offer a selection between FXS+noSMP and noFXS+SMP builds.
I could just modify the .dtsi file and/or the buildsystem could be updated to offer the mentioned options, but I'd rather like to be able to modify / edit / change the kernel command line / bootargs on a flashed router, without the need to reflash it.

So far I managed to define a slightly different bootargs var in uboot over a serial connection:
set bootargs 'console=ttyLTQ0,115200 mem=61M vpe1_load_addr=0x83e00000 vpe1_mem=2M maxvpes=1 maxtcs=1 nosmp'

... but LEDE just ignores mem=61M and still reports:
[ 0.000000] Kernel command line: console=ttyLTQ0,115200 mem=62M vpe1_load_addr=0x83e00000 vpe1_mem=2M maxvpes=1 maxtcs=1
root@OpenWrt:~# cat /proc/cmdline console=ttyLTQ0,115200 mem=62M vpe1_load_addr=0x83e00000 vpe1_mem=2M maxvpes=1 maxtcs=1

How can I get LEDE to use the bootargs var provided by me, instead of the one integrated at build time?

And as a side question: How can I modify the uboot environment from a running LEDE?

So far I installed the packages uboot-envtools and kmod-mtd-rw but nothing works:

root@OpenWrt:~# cat /etc/fw_env.config
/dev/mtd2 0x0000 0x20000 0x20000

root@OpenWrt:~# fw_setenv bootcmd console=ttyLTQ0,115200 mem=61M
Can't open /dev/mtd2: Permission denied
Error: can't write fw_env to flash

root@OpenWrt:~# insmod mtd-rw.ko i_want_a_brick=1

root@OpenWrt:~# mtd unlock mtd2
Could not open mtd device: mtd2
Could not open mtd device: mtd2

root@OpenWrt:~# fw_setenv bootcmd console=ttyLTQ0,115200 mem=61M
MTD erase error on /dev/mtd2: I/O error
Error: can't write fw_env to flash

Of course I tried that only after getting fw_printenv working:

root@OpenWrt:~# fw_printenv
addconsole=setenv bootargs $bootargs console=$consoledev,$baudrate
addeth=setenv bootargs $bootargs ethaddr=$ethaddr
addip=setenv bootargs $bootargs ip=$ipaddr:$serverip::::$netdev:off
addmachtype=setenv bootargs $bootargs machtype=VGV7510KW22
bootargs=console=ttyLTQ0,115200 mem=61M vpe1_load_addr=0x83e00000 vpe1_mem=2M maxvpes=1 maxtcs=1 nosmp
bootcmd=bootm ${kernel_addr}
write-uboot-nor=protect off 0xB0000000 +$filesize && erase 0xB0000000 +$filesize && cp.b $fileaddr 0xB0000000 $filesize

(I removed a few unnecessary lines)

BTW: I accidentally mixed up fwtools with uboot-envtools and uninstalled fwtools. This completely bricked my LEDE, partly because the package base-files depends on fwtools and all that stuff was marked as deleted in the file system overly.
I couldn't even access a shell over the serial console. All I got after most of the usual boot/kernel log was repeated messages like this: Failed to execute Please press Enter to activate this console.
Luckily FAILSAFE MODE was working and I could've just cleared the overly ("firstboot - reset settings to factory defaults").
Instead I opted to manually compare /rom/ with /overlay/upper/ and fix all the wrongly removed/modified files etc. Now everything seems to be working again.

IMHO users should not be able to uninstall any essential packages with opkg...
Should I report this as a bug? Or make a feature request?


Did you figure out how to modify the parameters appearing in /proc/cmdline (without rebuilding)?

Depends on the device. Many ignore the passed boot args completely, or "munge" then significantly.

1 Like

No I did not.
But I have build an image that should work -> Proposal: do *not* ignore uboot-env-args if e.g. ubenvar "respect_ubenv=yes" is set
... but after compiling I lost my motivation and haven't found it again, thus the image is still untested and only in my linux vm... :-/

1 Like

Understandable. In the end I solved by other means the problem that brought me here. My problem was that with the Raspberry Pi device I was using, some proprietary Broadcom software calls the openwrt kernel - functioning as bootloader. Unfortunately, even though there was no change in the Broadcom hardware, they changed the loader software in a way that broke back compatibility with existing software. So the problem was solved by using older versions of the Broadcom loading software.

any simple way to do this on image containing 2.6 kernel? i've extracted fw, decompressed kernel from lzma, i can find cmdline inside with hexeditor. would it work to just change it, compress with lzma and repack firmware, or some checksum algo exist for vmlinux images during creation?

So, I've tried my image from over half a year ago and my bootargs set in uboot are still ignored :frowning:

But now in the process of updating my buildenvironment I stumbled over this line in .../openwrt/target/linux/lantiq/config4.9: # CONFIG_MIPS_CMDLINE_FROM_BOOTLOADER is not set

Is this the culprit?

Likely... check for ATAG and EXTEND also ( dump the full kernel .config if you have to )... make kernel_menuconfig is generally the simplest way to find out.

just deleted my old build folder to start new with a current image. :-/
but EXTEND definitely wasn't set either and at least one other related line.

in the meantime I found this: Kernel command line bootargs being ignored despite kernel_menuconfig settings

It seems this will not work unless I manually modify some kernels files to keep an override from being compiled into the kernel.

Yes, I've had to inline mod those with a custom script. ( correction... i had to inline mod USB options but for ipq806x... ATAG/EXTEND/BOOTLOADER in config-4.14 is adequate )...

Seems like EXTEND and/or ATAG were never properly enabled. Let me know how to select your target/board in the buildroot and i'll see if I can help out.

Thanks for the help/offer but it appears I didn't read your post till its end :face_with_hand_over_mouth: and in the meantime I managed to get it kinda working how I want to. :slight_smile:

CONFIG_MIPS_CMDLINE_FROM_BOOTLOADER was the missing culprit and after seeing that / in make kernel_menuconfig enables me to search for config options I found it under "Kernel type -> Kernel command line type -> Bootloader kernel arguments if available" and NOT under "Kernel Hacking -> ???" where the built-in command line is set and so on.... :confused:

Now I can define my own bootargs in u-boot but the built-in ones are attached at the end (I need to replace them) and this behavior led me to the following "test".
First define this in the u-boot environment:

setenv bootargs console=ttyLTQ0,115200 - --

Then check in openwrt:

root@OpenWrt:~# fw_printenv | grep -i bootargs=
bootargs=console=ttyLTQ0,115200 - --

root@OpenWrt:~# cat /proc/cmdline
console=ttyLTQ0,115200 - -- console=ttyLTQ0,115200 mem=61M vpe1_load_addr=0x83e00000 vpe1_mem=2M maxvpes=1 maxtcs=1 nosmp

root@OpenWrt:~# cat /proc/cpuinfo | grep -i process
processor               : 0
processor               : 1

Note that /proc/cmdline still contains the built-in bootargs after the - but nosmp is ignored and both cores are available.

Linux (the kernel) stops processing further arguments in bootargs/cmdline after the first lonely hyphen-minus (-) and passes the rest to init. I couldn't find any such feature for init but i added two - just in case.

Thus one and the same image can be used with both CPU cores or with only one core but with the FXS stuff.

The rest of my goal is to get this change into owrts src so all future builds for the o2 6431 router+modem let the user modify the bootargs in the u-boot environment (when installed and configured).

I just need someone with such a router to test if this change is BRN-boot-safe and no faulty bootargs get passed to the kernel when the original BRN-bootloader is still installed.

One of my o2 6431 routers still has an uninitillized u-boot environment and I'll test if my change has any negative consequences there.

1 Like

[thinking out loud]
from what i can see... this would be "traditionally" bundled as two images... or perhaps something similar to luci-app-altreboot ... but i've never seen targets/device specific luci code... ( as part of a default image build )...

does seem kinda wastefull to have to do ^ for a few characters on a cmdline... but at the same time... you can kinda see where the philosophy around keeping the cmdline static / oem comes from...

maybe hnyman, jeff. jow or one of the other low level devs can give you some input on the "politics of the cmdline" with regard to official acceptance before you burn too much time... ( maybe send an [RFC] to the mailing list as feedback on these things here is often minimal )

oh, I didn't think/"dream" that far. Since oWRT makes the u-boot MTDs read-only anyway I meant modifying the u-boot environment from within u-boot over rs232 as "modifiable by the user" (OT: why is modifying written with a y but not modifiable?).

But yeah, thinking about it I have to agree that a third image (BRN, NOR, NOR smp+noFXs) is the better solution not only because of the cmdline philosophy but also because there are several packets a SMP+noFXS image could leave out in comparison to the default noSMP+FXS images.

The ability do modify the bootargs/cmdline could be an added bonus for this third image then.

Maybe I'll RFC the ml regarding that.

If there's a reason, it's simple to remove the read-only flag (devices with dual-firmware setups often need uboot-env write access), but see

1 Like

I know about that flag in the .dts(i) file and until half an hour ago I would've said there's not enough of a reason for making an rw change there.

Now I think I have a reason and a mystery on my hands.

I did that with an image with this kernel config:

grep -i cmd /target/linux/lantiq/config-4.9

but after the update I got this:

Warning: Bad CRC, using default environment
bootcmd=bootp; setenv bootargs root=/dev/nfs nfsroot=${serverip}:${rootpath} ip=${ipaddr}:${serverip}:${gatewayip}:${netmask}:${hostname}::off; bootm
:~# cat /proc/cmdline
console=ttyLTQ0,115200 mem=62M vpe1_load_addr=0x83e00000 vpe1_mem=2M maxvpes=1 maxtcs=1 nosmp
cat /proc/cpuinfo
system type             : xRX200 rev 1.1
machine                 : o2 Box 6431
processor               : 0
cpu model               : MIPS 34Kc V5.5
BogoMIPS                : 332.54
wait instruction        : yes
microsecond timers      : yes
tlb_entries             : 16
extra interrupt vector  : yes
hardware watchpoint     : yes, count: 4, address/irw mask: [0x0ffc, 0x0ffc, 0x0ffb, 0x0ffb]
isa                     : mips1 mips2 mips32r1 mips32r2
ASEs implemented        : mips16 dsp mt
shadow register sets    : 1
kscratch registers      : 0
package                 : 0
core                    : 0
VPE                     : 0
VCED exceptions         : not available
VCEI exceptions         : not available

So far so bad - it looks like the default built in cmdline is still getting in somehow.
The obvious next step was creating a proper environment in uboot over a serial console with bootargs=cmdline=ttyLTQ0,115200.
After having done that, most commands from above gave the expected proper output (two CPUs, fw_printenv) except for

:~# cat /proc/cmdline
console=ttyLTQ0,115200  console=ttyLTQ0,115200

Now cmdline behaves just like in post #11 with concated "bootargs"+"built-in-cmdline".

Whats going on here? Why didn't the first cat /proc/cmdline contain two sets of console=tty..., why did it contain the original built-in one at all?
Doesn't oWRT restart itself and/or reboots the router during a system upgrade?
That's the only explanation I can think of why the cmdline didn't change at all during the update.
Unfortunately I only got this idea after I had already created a proper uboot environment, thus I can't test what a simple power cycling would have done. :-/

not sure if this is of use...

some ( dynamic ) sysupgrade uboot env handling...

cat target/linux/ipq806x/base-files/lib/upgrade/

you may need to add fw_printenv to COPY_BINS ( ) and pay attention to the lock creation...

the non-dynamic alternative is hardcoding in multiple dts

it does restart... and another ( slim ) possibility is uboot recreating it's default env... which I think is half the reason for dts based cmdlines or custom uboots...

and there are these which are probably not be relevant...

random cmdline kopts
cat <<EOF >/tmp/atagenable

Okay, after having let the information from you two (@wulfy23, @slh) sink in some more I've come to the conclusion that a third image is in fact completely unnecessary as long as future release of the default images are modified in such a way that they work in all of the following setups:

  • BRN-boot, 1st flash of brn-squashfs-factory.bin without working uboot environment. I assume this is the most used first stage for the o2 6431 / vgv7510kw22 router.
  • BRN-boot, update-flash nor-squashfs-sysupgrade.bin still without ubootenv.
  • u-boot, only ever nor-sysupgrade images, without ubootenv and possible remains from when BRN-boot was still flashed (dunno if this can happen. couldn't happen for me because I used a mostly empty 16MiB image to overwrite the whole flash once with uboot, oWRT and a copy of the original board_config partition -> all possible remains where the ubootenv partition is were overwritten).
  • u-boot, only ever nor-sysupgrade images, with working ubootenv (w/w/o bootargs defined). This is the setup I'm using.

To achieve this the following conditions must/should be met (I think):

  1. integration of @slh's patch from #14.
    As I understand it this would enable the uboot-envtools (fw_printenv, fw_setenv) to create an ubootenv automatically the first time they're used? Or does this somehow trigger an installed u-boot to automatically create the proper environment? Kinda confused what the purpose is if that's not the case. Or is this just a patch to let fw_setenv know where to write on this specific device, when no ubootenv exists yet?
    Would this do something to routers still with BRN-boot?
  2. The user needs to be able to write to the ubootenv partition. Be that with insmod mtd-rw i_want_a_brick=1 (doesn't work AToW) or with a change in VGV7510KW22NOR.dts.
    Since all partitions would remain write protected (most of the time) either way in BRN setups (VGV7510KW22BRN.dts) most of my other concerns are baseless anyway, aren't they? :wink:
  3. Some kernel compile parameters concerning the cmdline/bootargs need to be changed but this can't mess with setups without a working ubootnev or wonky remains.

CONFIG_CMDLINE="console=ttyLTQ0,115200 mem=62M vpe1_load_addr=0x83e00000 vpe1_mem=2M maxvpes=1 maxtcs=1 nosm" should stay in its predefined way as it is now.
Dunno if the CMDLINE DTB entries are of concern...

  1. The normal images with noSMP+FXS shouldn't be much larger then a theoretical default image without the necessary packages for telephony. That's already the case imho.
  2. Overriding the built-in cmdline with console=ttyLTQ0,115200 must disable all non-SMP VPE activity. I think I'm still getting some unnecessary action there:
some kernel boot messages with vmmc/vpe stuff?!
[    0.504545] lantiq,vmmc 1f103000.vmmc: requested GPIO 465
[    0.509865] lantiq,vmmc 1f103000.vmmc: requested GPIO 493
[    0.515332] lantiq,vmmc 1f103000.vmmc: requested GPIO 492
[    0.520782] lantiq,vmmc 1f103000.vmmc: reserved 1MB at 0x03b00000
@/after 59MiB?

[    0.649305] No VPEs reserved for AP/SP, not initialize VPE loader
[    0.649305] Pass maxvpes=<n> argument as kernel argument
[    0.660678] No TCs reserved for AP/SP, not initializing RTLX.
[    0.660678] Pass maxtcs=<n> argument as kernel argument
[    0.673549] Crashlog allocated RAM at address 0x3f00000
[    0.679552] workingset: timestamp_bits=30 max_order=14 bucket_order=0
@/after 63MiB?

Any further hints/ideas an how I can make sure none of the usual setups/constellations can lead to oWRT booting with an unintentional cmdline?

Open questions (from my perspective):

  1. Is the area where the ubootenv is placed erased regardless of the way you install uboot+oWRT?
    The ToH page explains a way to install both over a serial connection and step three of "Install OpenWrt from u-boot" is "Erase the flash area where u-boot and it's environment are stored" but the command looks like it only erases the area/partition for oWRT???
  2. Is it actually problematic if it's not erased?
  3. (pure curiosity) Why do neither .dts-file mention/define the board_config @ 0xfe0000 (16MiB - 128KiB)?
  4. Whats happening at 5. (see above)?
  5. Am I missing a downside to making the ubotenv partition writable from within oWRT?
  6. Should I just try to introduce the necessary patches with a pull request in github? (have I done enough preliminary work?
  1. I personally have no experience with BRN... let's assume for a second that uboot itself is not overwritten ( perspectiveA )... most of the uboot-env stuff is super simple in that case ( apart from the ethics / actual value testing ), your specific needs are probably compounded if the stock bootloader is modified / augmented / etc ( perspectiveB )... so thinking about it from perspectiveA and getting over that obstacle/s... then tackling any further caveats that arise from perspectiveB.... would probably assist others / you, to understand and resolve things....

  2. secondly two random thoughts;

  • Many "dev-boards" pass dts and/or boot.scr ( so i assume they all have atag enabled? depending on your overall "target" ( other boards ) setup... perhaps this is an alternative to multiple images and/or a reference...

I've asked about this in my thread re:usb-boot and got almost zero feedback... ( in my case... all the other target>boards have ATAG ignore... plus i'd like to add usb to the kernel ~not absolutely required~ ... from what I can see the former can only be done to all target>boards but i've yet to try something parallel to generic-subtarget-folder... the latter is similar but kind of needs a menuconfig for that board>subboard~separate-image~... I suppose potentially the github UEFI-PR might show some approaches to this )

Let me have a go at answering your questions with my limited knowledge;

  1. It's up to you, but i'd assume that for most uses... this is the fastest, cleanest way to populate a "known" config... bare in mind that some uboot's will zap~auto-recover their env... which means that oem interference troublesome, thus "zapping" it yourself ( or via the uboot you've compiled ) is a "diligent integrity check". In my case... most of ipq806x boards work with oem many are dual-firmware which semi-demands known compatibe env... so "zapping" as a means of efficiency is not an option.

  2. Did my best to "guestimate" that above...

  3. Makes sense to have in the .dts... but I think u-boot itself has it compiled in... and on-device... pre-dts tools still make use of /etc/fw_env.config ( uboot-env-tools )

  4. Don't understand the question... possible answered above

  5. The downside is on a) dual-firmware devices... or oem routines that might expect something and their factory restore does not handle it etc. etc.... b) misconfiguration or corruption makes the device zombified... and where there is no serial... read only serial... hard to get to serial.... then the whole world is effected c) relying on frequent utilisation of this has flash wear implications.

  6. It's can't hurt... and for input sake, I really hope you get some constructive advice and input to make this work. Most core-developer time is spent on "the latest and greatest" and sadly not enough on "massaging key contributions" to completion and supporting those that deserve help. Huge shoutout to @jow @adrianschmutzler @jeff @bobafetthotmail @slh and the five-ten other people i've forgotten who invest their time and energy supporting the greater community at it's roots, where it's sorely needed.

Not sure I understand you correctly but the o2 6431 device (and i think all other devices with brnboot) doesn't have (any) uboot installed and the ubootenv is empty when one installs uboot & openwrt this way.

Now first I have to say that I'm going to treat this as a git training for myself with a new o2 6431 box in a few days and just try it (f*ck this whole Linux-hyper-nerd philosophy of turning every good and useful tool into a console-only based anti-teaching user enema that hurts the eyes and only uses 1% of our visual capabilities. sorry, rambling...)

regarding your two thoughts: I think I don't understand them because the background periphery responsible of thinking them up is unavailable to me. :wink:
And several oWRT buzzwords are not in my in-brain dictionary:

  • dev-boards - ??
  • dts - the .dts file from/for a specific image target (in my case two. NOR for sysupgrade / uboot and BRN for the brnbootloader)
  • boot.scr - what? a windows screensaver? :grin:
  • atag - attach tags CONFIG argument in the kernel config?

OT in regard to usb-boot: Sounds like a good idea but isn't this only possible/useful on SBCs and not on normal routers which are the target for oWRT? How should eg. uboot boot a Linux kernel from an USB attached thumbdrive? with kexec you'd need a kernel that does that on the router.
And for development I've seen people just use a NFS root plus maybe tftp.

To your answers (I had hoped some o2 6431 users would chime in):

  1. as stated above there is no uboot unless the owner installs it on these routers and I will test this with the new router.
  2. --
  3. nonono not the uboot env. i mean the factory board calibration, MACs and so on (board_config).
  4. I referenced my point 5. with "some kernel boot messages with vmmc/vpe stuff?!".
  5. it's not a dual-firmware device with uboot. with brn-boot there are two bootable images on the flash afaik. This router is unusable without oWRT and the serial port is easily accessible. -> I'm interpreting this as "most likely no other downsides".
  6. will try it.

= Rpi, Bpi, Olinuxino, Lime, Orange... etc. ( generally pass a device tree separately... )

Likely means your /etc/fw_env.config is missing, malformed or incorrect... But may also mean that it's blank and no defaults were specified on uboot compile... ( fw_printenv / fw_setenv don't create full environments normally... this is in the uboot makefiles / source )

( p.s. my dual boot is working great now... :nerd_face: )