Rpi4 < $(community_build)

Finally happened again today,

Here's the output of logread -e collectd > collectd.log

Looks like there have been multiple respawns of collectd (I count 13) - it seems the last respawn before the more than 2 logs didn't exit normally, and is missing the following:

daemon.err collectd[2751]: Exiting normally.
daemon.err collectd[2751]: collectd: Stopping 2 read threads.
daemon.err collectd[2751]: collectd: Stopping 5 write threads.

These preceded the prior collectd restarts, but not for the one that appears to have multi-spawned sqm_collectd.sh

Here's the output of uci show luci_statistics | grep dns

luci_statistics.collectd_dns=statistics
luci_statistics.collectd_dns.enable='1'
luci_statistics.collectd_dns.Interfaces='eth0'
luci_statistics.collectd_dns.IgnoreSources='127.0.0.1'
luci_statistics.collectd_processes.Processes='uhttpd dnsmasq dropbear'

It does seem that the errors I saw thrown previously by the dns plugin were bad timing and possibly unrelated as they're not thrown here.

Let me know if there's anything else I can dig into!

Cheers

edit: grammar...

1 Like

Just pulled this down (with multiple sqm_collectd.sh's running) and restarted collectd:

sh /etc/init.d/collectd stop
sh /etc/init.d/collectd start

Can confirm this killed the other instances :+1: logging has resumed

1 Like

freikfunk feed has been removed from master I believe... this probably wont be present in newer (factory) distfeeds.conf files...

as these are carried across within sysupgrade data... users of existing builds are advised to disable the feed in '/etc/opkg/distfeeds.conf' preferrably prior to any upgrades (or if you restore a backup) as it will likely bork autorestore functionality;

#src/gz openwrt_freifunk https://downloads.openwrt.org/snapshots/packages/aarch64_cortex-a72/freifunk

Hi.
I've been pushing my rpi 4 a little too much with multiple jobs all over the day and I would like to overclock it.
However, when I try to do it, it appears the frequency does not change.
Isn't it enough to set in /boot/config.txt the lines:
over_voltage=3
arm_freq=1600

Thanks in advance

1 Like

due to the various risks and complexities involved with overclocking...

on this build... the preferred method to 'overclock'(and 'underclock') is by tweaking the governor ( runtime within OS )...

/root/wrt.ini

POWERPROFILE="quickest"

try the above... ( check there is not a value already assigned i.e. a line that is uncommented... I may have set 'quick' as the default recently... cannot remember )

( in which case change that or duplicate and comment it out )

after changing this setting to apply without rebooting run;

/bin/rpi-perftweaks.sh

fyi, as your sending your collectd stuff to a server... technically you could/should do without the 'persistentlucistatistics' feature...

alas... I haven't cleanly checked/implemented a clean way to do this currently... i'd assume adding SERVICESDISABLED="persistentlucistatistics" to wrt.ini and commenting out the existing line in /etc/crontabs/root

# 0 */6 * * * /etc/init.d/persistentlucistatistics psave
/etc/init.d/persistentlucistatistics disable
/etc/init.d/cron restart

would be the logical way to do this... but probably better to not bother for a few builds (or at all if the workaround is ok)... I may get the chance to test/tidy over the next few builds...

basically... what these do is

  • copy /tmp/rrd to persistent storage in case of power loss ( @neil1 made me aware of this condition ) prior to that... there was no constant 'build-related' restarting of collectd...
  • some other stuff that tries to transparently retain luci_statistics and nlbwmon data over reboots and upgrades (messy as carp but I'm a little scared to touch it...)

hoping the 'end-existing' workaround largely resolves the fundamental issue for now ...

you can disable that cron script... although the firstboot logic may put it back each upgrade...

1 Like

hwrng seems to be working very nice!

The next overclocking options will work if you have a good power source and enough cooling, of course:

# Overclocking to 1.75GHz
over_voltage=2
arm_freq=1750

# Overclocking to 2Ghz
over_voltage=6
arm_freq=2000

# Overclocking to 2.174GHz
over_voltage=6
arm_freq=2147
gpu_freq=750

Mine is running at 2Ghz for different reasons, since 8 weeks ago. If what you need is a quick ramp up of CPU frequency just go with @anon50098793's governor tweaks.

1 Like

Is this installing rng-tools? Or, is it just executing echo 3800 > /proc/sys/kernel/random/write_wakeup_threshold? Pure curiosity.

1 Like

yeah... it's a bit of an odd one... rngd/rng-tools@/dev/urandom seems to be the sweet spot... ( i'm sure something whacky gets switched out at a lower level )

$ecmd "hardcode workaround setting to urandom seems to sync all 3"
uci set system.@rngd[0].device='/dev/urandom' #NOPE uci del system.@rngd[0].device='/dev/urandom'
uci commit system
/etc/init.d/rngd enable; /etc/init.d/rngd restart
uci show system | grep rng
system.@rngd[0]=rngd
system.@rngd[0].enabled='1'
system.@rngd[0].device='/dev/urandom'

I will swap /dev/urandom with /dev/hwrng in the RPi4 and disable default urngd if you are up for a quick test. :wink:

yup... pretty sure that's what I attempted first... but for some odd reason... doesn't seem to kick in properly...?

(edit: didn't test after this tho... may have helped the situation?)

i2c / serial uC

i've just added i2c kmods and serial drivers for common ttl chipsets...

i'm running an hd44780 over serial cmds to an arduino... and i've got an i2c-expander on i2c-1...

this stuff should probably go in it's own thread if there is interest ( cmds over serial are doable on most routers... i2c expanders are not far off )

worth mentioning in case anyone else has a use for some relays / display / rtc abstractions etc...

Just an FYI of my experience on the RPi 4B so far.

  • I'm using an Rpi 4B w/ 8 GB RAM with two USB/Ethernet adapters (one is RTL8153 and one is AX88179, so 3 Ethernet ports in total) and a 32 GB A1/U1/C10 micro SD card for the code. And a 5V/3A power supply; plenty to support two USB/Ethernet adapters.

  • I am using the /dev/sda2 resizing ideas from this page: https://www.zahradnik.io/raspberry-pi-as-a-home-router

  • First I installed Snapshot 25730 r15963 all worked well for a day or two.

  • Both USB/Ethernet adapters worked from the start. No additional installs needed.

  • BUT once the power died (life here in the country) OpenWRT never came back up. It served the 192.168.1.1 on Eth0 but I could not SSH in, could not access the HTTP page nor ping Eth0. That was with and without the two USB/Ethernet adapters.

  • I reinstalled the snapshot (dd to /dev/sda, resize sda2, etc).

  • That started up fine, but it would not take the backup file made earlier....

  • So I reconfigured everything, then as a test I pulled the plug.

  • Dead again.

-Ok this is snapshot code so move on. I next applied Snapshot 25737 r16121.

  • Everything works again (actually a bit better than before).

  • I then pulled the plug again to see what happens.

  • Same problem, Eth0 served 192.168.1.1, but no ping, SSH or HTML possible.

  • I reinstalled everything again (still can't use backups) and it too could not recover from a power failure.

My old TP Link AC1750 has no such problem with OpenWRT.

Let me know if I neglected something so this could recover from a power failure but don't make a big deal of it. I'll wait for a few more revisions before I try again.

Thanks for all the good work folks!

1 Like

Thanks @anon50098793

Seems to be working so far, I've seen some further unclean exits crop up from collectd, but still only ever seeing one version of sqm_collectd.sh running every time I've checked.

I'm trying to keep my build as close to yours as I can, so I don't have to think/remember too much when upgrading (I've made that mistake far too many times!), so I'll leave out disabling persistentlucistatistics changes out for now.

1 Like

Just wanted to add the opposite experience,

I’ve been running these builds for over a year with multiple power failures / unplugging the pi while booted.

I’ve not had any forced power failure cause a reboot problem, l may be in the minority however.

Have you tried burning the image with no modifications (partition size etc) and tried to reproduce?

1 Like

sorry, i missed your post until right now... and first of all thankyou for posting your experience / feedback...

background
  1. due to alot of custom logic within the build... resizing partitions poses quite a few potential break points...

  2. i have alot of experimental/disabled code within the build... one of those expands the rootfs partition to use the whole sdcard... part of the issue I have with fully promoting/finishing this option is if a user then fulls up the root partition with data... they typically then need to migrate that data outside of typical backup / keep settings methods... having a spare usb drive block mounted and pointing all your service data there totally avoids this complexity... is immediate, predictable, extensible... etc. etc.

to investigate

so... if you wish to keep trying first thing you should exclude is cmdline issues...

cat /boot/cmdline.txt
blkid $(findmnt /) #beforeresize
blkid $(findmnt /) #afterresize

after that... it's probably your resize proceedure conflicting with fsckparts on the command line...

  • you pulling the power will activate fsckparts if it is enabled... and it's not happy with what it's finding... could be a bug on my side or yours... but you'll probably need a serial console to actually see whats happening...

in short... as stated before... the 'recommended' method to obtain more storage with this build is to use a usbstick...

( that said... I am ready and willing to further assist you to debug and possibly test the build in rootfsexpand feature [trying again in a few months likely wont help but actively testing and collaborating will]... there is no doubt it will lead to better robustness more generally anyway )

for reference here is what I run;

########## uci show fstab
fstab.@mount[2]=mount
fstab.@mount[2].target='/usbstick'
fstab.@mount[2].uuid='d088ab74-76b6-4760-aa88-3d0234c06d48'
fstab.@mount[2].enabled='1'

wrt.ini / or any other service data params (optional)

RCSHDDIR="/usbstick"
RPI4QOS_RECLASSIFY_SH="/usbstick/reclassify.sh"

this is probably a more important separate issue...

for now... if you can next test this by only using a backup created on the same version... and try to restore from the command line with;

sysupgrade -v -r <backupname.tar.gz>

and post the output here...

also search the thread for 'sysupgrade-clean-backup.sh' and read any related comments around those posts... it's a wip but again polishing, collaborating and more testing of this is needed...

Ok folks,
Thanks for the advice.

So far I

  • dd'd a new copy of Snapshot 25737 r16121 to my micro SD card (I'll leave the USB stick matter for another exercise since it is not a high priority).

  • I did -not- expand the the second partition or do anything else to the SD card. I just booted from it in the RPi4B

  • I then attached and mounted a Samsung T5 SSD with the backup file.

  • I ran Wulfy23's sysupgrade -v -r <backupname.tar.gz>
    and that "worked". I say "worked" in the sense that it appeared to work; all my settings were back, and I could pull/kill power to the RPi4B and it would restart as desired.
    You folks were right that enlarging the partition was the root of my problem.

  • My next issue is that my eth1 and eth2 adapters get mixed up (eth0 is the port on the RPi4B board). I'll bring this up as a separate question below to keep things separate.

  • Let me say here for completeness that when I said yesterday that this Snapshot code would not take the backup file, that was from in LuCI. I did not try it from in an ssh/terminal at that time.

  • As requested, here is the result of the sysupgrade -v -r command:

root@rpi-dca632cd0f /29# ll /mnt
drwxr-xr-x    2 root     root          4096 Mar  6 04:42 ./
drwxr-xr-x   20 root     root          4096 Mar  6 04:42 ../
root@rpi-dca632cd0f /28# mount -t ext4 /dev/sda1 /mnt
root@rpi-dca632cd0f /29# ll /mnt
drwx------    7 1000     1000          4096 Mar  8  2021 ./
drwxr-xr-x   20 root     root          4096 Mar  6 04:42 ../

... misc files....

-rw-rw-r--    1 1000     1000        655360 Mar  8  2021 backup-rpi-dca632cd0f-2021-03-08.tar.gz
drwx------    2 root     root         16384 Feb 23 14:55 lost+found/

root@rpi-dca632cd0f /27# sysupgrade -v -r /mnt/backup-rpi-dca632cd0f-2021-03-08.tar.gz 
boot/cmdline.txt
boot/config.txt
etc/collectd.conf
etc/config/.servicestate
etc/config/acme
etc/config/adblock
etc/config/atftpd
etc/config/banip
etc/config/collectd
etc/config/darkstat
etc/config/ddns
etc/config/dhcp
etc/config/dropbear
etc/config/etherwake
etc/config/firewall
etc/config/fstab
etc/config/https-dns-proxy
etc/config/irqbalance
etc/config/luci
etc/config/luci_statistics
etc/config/mwan3
etc/config/network
etc/config/nlbwmon
etc/config/openvpn
etc/config/openvpn_recipes
etc/config/opkg
etc/config/pservice
etc/config/rpcd
etc/config/simple-adblock
etc/config/snmpd
etc/config/socat
etc/config/sqm
etc/config/system
etc/config/travelmate
etc/config/ucitrack
etc/config/uhttpd
etc/config/vpn-policy-routing
etc/config/watchcat
etc/config/wifi_schedule
etc/config/wireless
etc/crontabs/root
etc/custom/cumulative.txt
etc/dropbear/dropbear_ed25519_host_key
etc/dropbear/dropbear_rsa_host_key
etc/firewall.user
etc/group
etc/hosts
etc/inittab
etc/iscsi/iscsid.conf
etc/luci-uploads/.placeholder
etc/opkg/customfeeds.conf
etc/opkg/keys/0b26f36ae0f4106d
etc/opkg/keys/1035ac73cc4e59e3
etc/opkg/keys/2f8b0b98e08306bf
etc/opkg/keys/5151f69420c3f508
etc/opkg/keys/72a57f2191b211e0
etc/opkg/keys/792d9d9b39f180dc
etc/opkg/keys/9ef4694208102c43
etc/opkg/keys/b2d571e0880ff617
etc/opkg/keys/b5043e70f9a75cde
etc/opkg/keys/c10b9afab19ee428
etc/opkg/keys/dace9d4df16896bf
etc/opkg/keys/dd6de0d06bbd3d85
etc/opkg/keys/de1e2a01d2bba155
etc/opkg/keys/f94b9dd6febac963
etc/passwd
etc/php7/20_curl.ini
etc/php7/20_fileinfo.ini
etc/php7/20_iconv.ini
etc/php7/20_json.ini
etc/php7/20_session.ini
etc/php7/20_sockets.ini
etc/php7/20_sysvmsg.ini
etc/profile
etc/profile.d/50-openvpn-easy-rsa.sh
etc/profile.d/55-lucisshon.sh
etc/profile.d/97-sysinfo.sh
etc/profile.d/99-updatecheck.sh
etc/profile.d/ashprofile.sh
etc/profile.d/sys_bashrc.sh
etc/profile.d/zshddir.sh
etc/quagga/ospf6d.conf
etc/quagga/ospfd.conf
etc/quagga/zebra.conf
etc/rc.local
etc/shadow
etc/shells
etc/shinit
etc/sysctl.conf
etc/sysupgrade.conf
etc/tmux.conf
etc/uhttpd.crt
etc/uhttpd.key
restorefiles/config-firstboot-post-202103060443/acme
restorefiles/config-firstboot-post-202103060443/adblock
restorefiles/config-firstboot-post-202103060443/atftpd
restorefiles/config-firstboot-post-202103060443/banip
restorefiles/config-firstboot-post-202103060443/collectd
restorefiles/config-firstboot-post-202103060443/darkstat
restorefiles/config-firstboot-post-202103060443/ddns
restorefiles/config-firstboot-post-202103060443/dhcp
restorefiles/config-firstboot-post-202103060443/dropbear
restorefiles/config-firstboot-post-202103060443/etherwake
restorefiles/config-firstboot-post-202103060443/firewall
restorefiles/config-firstboot-post-202103060443/fstab
restorefiles/config-firstboot-post-202103060443/https-dns-proxy
restorefiles/config-firstboot-post-202103060443/irqbalance
restorefiles/config-firstboot-post-202103060443/luci
restorefiles/config-firstboot-post-202103060443/luci_statistics
restorefiles/config-firstboot-post-202103060443/mwan3
restorefiles/config-firstboot-post-202103060443/network
restorefiles/config-firstboot-post-202103060443/nlbwmon
restorefiles/config-firstboot-post-202103060443/openvpn
restorefiles/config-firstboot-post-202103060443/openvpn_recipes
restorefiles/config-firstboot-post-202103060443/opkg
restorefiles/config-firstboot-post-202103060443/pservice
restorefiles/config-firstboot-post-202103060443/rpcd
restorefiles/config-firstboot-post-202103060443/simple-adblock
restorefiles/config-firstboot-post-202103060443/snmpd
restorefiles/config-firstboot-post-202103060443/socat
restorefiles/config-firstboot-post-202103060443/sqm
restorefiles/config-firstboot-post-202103060443/system
restorefiles/config-firstboot-post-202103060443/travelmate
restorefiles/config-firstboot-post-202103060443/ucitrack
restorefiles/config-firstboot-post-202103060443/uhttpd
restorefiles/config-firstboot-post-202103060443/vpn-policy-routing
restorefiles/config-firstboot-post-202103060443/watchcat
restorefiles/config-firstboot-post-202103060443/wifi_schedule
restorefiles/config-firstboot-post-202103060443/wireless
restorefiles/config-firstboot-pre-202103060442/acme
restorefiles/config-firstboot-pre-202103060442/adblock
restorefiles/config-firstboot-pre-202103060442/atftpd
restorefiles/config-firstboot-pre-202103060442/banip
restorefiles/config-firstboot-pre-202103060442/collectd
restorefiles/config-firstboot-pre-202103060442/darkstat
restorefiles/config-firstboot-pre-202103060442/ddns
restorefiles/config-firstboot-pre-202103060442/dhcp
restorefiles/config-firstboot-pre-202103060442/dropbear
restorefiles/config-firstboot-pre-202103060442/etherwake
restorefiles/config-firstboot-pre-202103060442/firewall
restorefiles/config-firstboot-pre-202103060442/fstab
restorefiles/config-firstboot-pre-202103060442/https-dns-proxy
restorefiles/config-firstboot-pre-202103060442/irqbalance
restorefiles/config-firstboot-pre-202103060442/luci
restorefiles/config-firstboot-pre-202103060442/luci_statistics
restorefiles/config-firstboot-pre-202103060442/mwan3
restorefiles/config-firstboot-pre-202103060442/network
restorefiles/config-firstboot-pre-202103060442/nlbwmon
restorefiles/config-firstboot-pre-202103060442/openvpn
restorefiles/config-firstboot-pre-202103060442/openvpn_recipes
restorefiles/config-firstboot-pre-202103060442/pservice
restorefiles/config-firstboot-pre-202103060442/rpcd
restorefiles/config-firstboot-pre-202103060442/simple-adblock
restorefiles/config-firstboot-pre-202103060442/snmpd
restorefiles/config-firstboot-pre-202103060442/socat
restorefiles/config-firstboot-pre-202103060442/sqm
restorefiles/config-firstboot-pre-202103060442/system
restorefiles/config-firstboot-pre-202103060442/travelmate
restorefiles/config-firstboot-pre-202103060442/ucitrack
restorefiles/config-firstboot-pre-202103060442/uhttpd
restorefiles/config-firstboot-pre-202103060442/vpn-policy-routing
restorefiles/config-firstboot-pre-202103060442/watchcat
restorefiles/config-firstboot-pre-202103060442/wifi_schedule
restorefiles/config-firstboot-pre-202103060442/wireless
restorefiles/config-wireless-preauto-20210306-0443
restorefiles/opkg.added.last
restorefiles/opkg.original.all
restorefiles/opkg.restore.added
restorefiles/service-dpata/banip/backup/banIP.bogon_4.gz
restorefiles/service-dpata/banip/backup/banIP.bogon_6.gz
restorefiles/service-dpata/banip/backup/banIP.darklist_4.gz
restorefiles/service-dpata/banip/backup/banIP.debl_4.gz
restorefiles/service-dpata/banip/backup/banIP.debl_6.gz
restorefiles/service-dpata/banip/backup/banIP.doh_4.gz
restorefiles/service-dpata/banip/backup/banIP.doh_6.gz
restorefiles/service-dpata/banip/backup/banIP.drop_4.gz
restorefiles/service-dpata/banip/backup/banIP.drop_6.gz
restorefiles/service-dpata/banip/backup/banIP.dshield_4.gz
restorefiles/service-dpata/banip/backup/banIP.firehol3_4.gz
restorefiles/service-dpata/banip/backup/banIP.threat_4.gz
restorefiles/service-dpata/banip/backup/banIP.voip_4.gz
restorefiles/service-dpata/banip/backup/banIP.yoyo_4.gz
tar: unexpected end of file
tar: short read
root@rpi-dca632cd0f /30# 
root@rpi-dca632cd0f /30# 
root@rpi-dca632cd0f /30# reboot

Just for interest, once the RPi4B was booting well from this snapshot, I added a third partition to the SD card. I left about 150 MB gap between partition 2 and 3 and ran 3 to the end of the SD card. I did this in fdisk, and formatted partition 3 ext4.

The RPi4B would not boot even thought partitions 1 and 2 were untouched.

Not a problem, just interesting.

1 Like

ok... so one of the reasons I created 'sysupgrade-clean-backup.sh' is to exclude certain stuff that is problematic...

with the new 'partuuid' based images... restoring an old (from a non-identical install) backup will insert a cmdline.txt that contains an incorrect root=PARTUUID=INCORRECTNUMHERE... ( well technically some older backups may have contained root=/dev/mmcblk0p2 which would restore/boot properly from the mmc slot )

i'll check the logic in that script over the coming week related to this... but in the interim... one workaround is to untar it on a hostpc remove cmdline.txt and retar it... or I believe if you change it back to root=/dev/mmcblk0p2 my logic doesn't touch it after that...(but you lose the ability to boot from usb)?