Rpi4 < $(community_build)

I'll keep you posted!
do i need to change stable to release?

1 Like

yeah... looks like you have 21.02.x so release is/was correct (sorry) but you can change to your liking at any time also... (might take 1 more min for github to propogate)

1 Like

and its working now!

1 Like

winner!!! :1st_place_medal: :1st_place_medal:

1 Like

Hi @anon50098793 firstly thanks for the amazing build and continuous releases just awesome!

Not sure if this is a bug but recently I'm getting this message in the log The net.core.rmem_max sysctl limit needs to be raised to least 1048576 in order to successfully set the desired receive buffer size! thought you might like to know just in case there are some adverse effects by not setting this correctly. Can't say I've noticed but hey I'm just an amateur in the networking world.

Keep up the great work!!!

1 Like

cheers... and good to get reports...

that's an odd artifact/message related to nlbwmon... i've set the sysctl's it's requested in a few different ways but the messages keep reappearing...

there are a few posts on the forum about it...

[root@dca632 ../_HTTPS-DNS-DEBUG 55°] logread | grep rmem
Wed Dec 22 21:02:20 2021 daemon.err nlbwmon[5542]: by the kernel. The net.core.rmem_max sysctl limit needs to be raised to
Wed Dec 22 21:02:25 2021 daemon.err nlbwmon[8108]: by the kernel. The net.core.rmem_max sysctl limit needs to be raised to
Wed Dec 22 21:02:26 2021 user.warn kernel: [   61.323646] 01-sysctl> /etc/sysctl.d/19-nlbwmon_rmem_max > '1048576'
Wed Dec 22 21:02:26 2021 daemon.notice procd: /etc/rc.d/S95done: net.core.rmem_max = 1048576

nlbwmon basically says XYZ is not XYZ > lowering

could have done something wrong (ok looking at the log... now seems I apply the setting a fraction too late) or could be an interface restart runaway sysctl or something... but yeah afaik... false positive...

will move it to the nlbwmon init.d script (or similar)... thankyou!


woohoo! :1st_place_medal: and it's gone!

[ /usbstick 57°] logread | grep rmem
Wed Dec 22 21:58:11 2021 daemon.notice procd: /etc/rc.d/S60nlbwmon: net.core.rmem_max = 1048576
Wed Dec 22 21:58:19 2021 daemon.notice procd: /etc/rc.d/S95done: net.core.rmem_max = 1048576
Wed Dec 22 21:58:19 2021 user.warn kernel: [   59.960211] 01-sysctl> /etc/sysctl.d/19-nlbwmon_rmem_max > '1048576'
1 Like

First time using Pi4 and your build, I have onsys:3.5.317-15 installed on a rpi4 8GB with a USB3 r8152 adaptor for WAN.

By default my connection speed tops out at around 510Mb/s.

When I cat /proc/interrupts I get this:

           CPU0       CPU1       CPU2       CPU3       
 11:       7819       5516       5742       5322     GICv2  30 Level     arch_timer
 18:       2759          0          0          0     GICv2  65 Level     fe00b880.mailbox
 21:          2          0          0          0     GICv2 153 Level     uart-pl011
 24:         45          0          0          0     GICv2 114 Level     DMA IRQ
 31:          7          0          0          0     GICv2  66 Level     VCHIQ doorbell
 32:      20654          0          0          0     GICv2 158 Level     mmc1, mmc0
 39:     114101          0          0          0     GICv2 189 Level     eth0
 40:      29685          0          0          0     GICv2 190 Level     eth0
 46:          0          0          0          0     GICv2 175 Level     PCIe PME, aerdrv
 47:     217562          0          0          0  BRCM STB PCIe MSI 524288 Edge      xhci_hcd

I was expecting eth0 to have affinity with a different core, so I manually set it to core 1 like this:

echo 2 > /proc/irq/39/smp_affinity
echo 2 > /proc/irq/40/smp_affinity

And now I'm hitting close to 1GB downloads - which is amazing! (thank you - great work!)

I thought IRQbalance was supposed to do this for me, is there something I need to do in order to active it? Or is the manual approach above the recommended solution?

Also when running speed tests CPU core 0 is hitting 99/100 percent, at the stock 1500Mhz, is it possible to enable an overclock to 1650Mhz or 2Ghz? I tried editing /boot/config.txt and adding:

over_voltage=6
arm_freq=2000

But this didn't work, HTOP is still reporting 1500Mhz.

howdy, merry christmas :wink:

will have to check my notes, but looks like a bug in /bin/rpi-perftweaks.sh (will take a look, thanks for the report!)

if you enable it, probably will, i don't use it, will render the above bug mostly irrelevant

maybe? i don't use config.txt overclock for a number of reasons, but there is a post or two in this thread or on the general forums about it so feel free to experiment...


as you are running 1G, I suggest you;


cp /bin/rpi-perftweaks.sh /bin/rpi-perftweaks.sh_SubZero
echo 'PERFTWEAKS_SCRIPT="/bin/rpi-perftweaks.sh_SubZero"' >> /root/wrt.ini
echo "/bin/rpi-perftweaks.sh_SubZero" >> /etc/sysupgrade.conf

then edit this file to your liking and share back with the rest of the community when happy...

( you can even just put exit 0 and enable irqbalance if thats what you want to do )

if you search this thread for PERFTWEAKS_SCRIPT you may find more recommendations or info for faster links...

Merry Christmas!

where did you set these?

echo 2 > /proc/irq/39/smp_affinity
echo 2 > /proc/irq/40/smp_affinity

so... it appears I manually/deliberately stopped assigning eth0 interrupts to different cores...

not sure why I did that, so will think about it some more... and search thread for reminders...

for a long time... (and from the top of my head) the build set 1st eth0 interrupt to core0 and second eth0 interrupt to core1, i'd recommend this for links over 600M as a beginning/baseline setting;

echo 1 > /proc/irq/<ETH0INT_0>/smp_affinity
echo 2 > /proc/irq/<ETH0INT_1>/smp_affinity

where do i set this?

echo 1 > /proc/irq/<ETH0INT_0>/smp_affinity
echo 2 > /proc/irq/<ETH0INT_1>/smp_affinity

see above...


if you are unfamilar with performance-optimization/script-editing then irqbalance may be a simpler option...

I want to run rpi-perftweaks.sh on boot automatically, how do i do that?
Thanks

already happens...

##### [ /usbstick 53°]# dmesg | grep perftweak | tail -n 1
[  179.868694] rc.custom> raspberrypi,4-model-b perftweaks /bin/rpi-perftweaks.sh [run]

fyi, for any (future) users with the seeed carrier... a kmod(.ko) is currently available here (21.02.1 only):

https://github.com/sergey-brutsky/openwrt-seeed-carrier-board

while it might be a little bit of a pain to install on this build for beginners worth mentioning... ( thanks sergey! )... when an official ipk becomes available for usb-net-lan78xx i'll add it to the build (even tho' it's unlikely there will be more than 1-2 seeed users afaik at most)

Go to SYSTEM / STARTUP / LOCAL STARTUP

and add those two commands to the end of the file, save, reboot.

On the snapshot that I installed, by default, core 0 is taking the load for just about everything. I guess this just isn't an issue for users with a connection of 0.5GB or less as a single core can cope. But on faster connections CPU affinity is critical.

I experimented a little with IRQ balance and packet steering, they work but not consistently, there seems to be a lot of volatility in how they distribute the load.

Manually assigning both eth0 interrupts to core 1 seems to be enough most of the time. But core zero is still hitting 100% when the data collection services are running.

My lazy fix is to overclock the Pi4 to give it just a little more headroom under load, but the better solution would be to manually distribute heavy services to specific cores. And given that the RPi4 is a known quantity, and the vast majority of users will be using a USB3 dongle, it seems reasonable to setup a default profile that just assigns all of this stuff at boot.

So my next question: is it possible to start services on specific cores?

PS I still can't find a way to overclock the Pi4 using OpenWRT - I'm a newb! ¯_(ツ)_/¯

1 Like

as there is no one size fits all solution, examples are provided in the build as well as hooks for using your own script as discussed above
( tried it several times already, what works for one does not necissarily work for others, and guess who ends up holding the pieces? )

you will find examples in the sample script already implemented
(nope: that was removed too search this thread for TASKSET or taskset)


more than happy to include anyone's suggested script within the build

this is currently the most practical way forward until broader user input or knowledgeable (succinct) test data is provided

That's fair, your solution of using user submitted scripts as a sort of 'selectable profile' is a very pragmatic approach that anyone can learn to adjust.

I'm working my way through your script now, I appreciate that you commented it but it's pretty hard going for a newb like me but I I'm getting there. I saw there's a bunch of commented out code in perftweaks that would implement CPU affinity and now I'm wondering why you disabled it?

Is there an overview guide of how your changes hang together? E.g where your scripts start in the boot process and how they are strung together. If that's a big ask please ignore.

The performance of your build is huge compared to the default snapshot builds I started with, I'm trying to understand what changes you've made purely from a performance standpoint. Is that all in perftweaks?

1 Like

thanks for the questions...

pretty much as above... spend around a month of my life on this... and in the end there;

  • was not enough skilled testers to provide the level of feedback / testing / future improvements
  • the support burden was massive, lots of questions about why ABC is not doing XYZ ( with little background information given )
  • as above, performance and performance optimization testing is a fickle thing... give 10 cooks a lasagne recipe and you will get 10 different things...

so I got a bit burnt out with it and said to myself: 'turn it all off except some reliable general tweaks, and allow the users to make their own changes'

its not absolute, and I do hope to one day to return to my better scripts, but for the medium term separate submitted user scripts is the most managable way to offer best performance for everyone without 10 fold factor in questions and problems compared to solutions and feedback offered...


it is all very basic... and i'll even fixup anything fancy if you want to submit/suggest something without the fancy variables and hooks... all tweaks boil down to around 10 simple commands...

almost everything in my build gets called out of rc.local > rc.custom > elsewhere

you can find almost everything with fgrep

fgrep -r rpi-perftweaks /etc/custom

there is one KEY element... this script is called in the background after a sleep of around 200 seconds... this is to allow for all services to startup and settle, otherwise RENICE and TASKSET may have no process to work on...

and because you went through the hassle of providing some quality observations... I dug up an old reference sample of the user configurable tweaks script for you if interested note: this does not discover eth0 irq numbers like the current one so wont work likely without fixes... useful for the servicecpuadjust() (taskset) you were asking about...


i don't think its really that major (but admittedly I have not really compared for a looong time)... has been discussed before in this thread... will link it here when I have some time...

essentially;

  • some teency config.txt changes or whatever
  • some sysctl's (but I don't think they really do much or at least I don't have a great understanding of what they really do... just seemed sensible / worth chucking in)
  • 2-5 days looking at the governor and limiting the threshold at which it throttles back cpu power... it's very agressive on these devices... minfrequ is set to maybe half of the frequency range as part of this
  • perftweaks (cpu affinity, process affinity, process renice, packet steering) all on or off or with minor variances depending on how far you go back / which build revision you look at
1 Like