NanoPI R2S is a great OpenWrt device

Does this built of openwrt work with luCI support? Where tot download the lastest built?
Am thinking of buying this board to setup as router. Want to upgrade the Archer C7 who does 13/13 mbit with open vpn. Hope the VPN speed will be higher with this device. Heart AES is usable with this board

Sorry, I am aware of the kernel bump. My question is how you managed to get 5.4.48 built and onto your R2S. I am not able to get a working build of @jayanta525’s latest repo. I know it’s being worked on, and that’s awesome.

In the mean time, was just curious if you could share any tips about how to get onto a newer kernel than 5.4.42. The only feature holding me back from switching over to full testing of my R2S is lack of SQM.

SQM really is just a collection of scripts, so you can install it by hand from git you will need to make sure that all required packets/modules are available manually as well.

Approximately 24 hours after receiving my NanoPi R2S, I am now up and running on it in place of my OpenWrt VM!

General usage looks good so far, but it doesn't appear to be keeping up with layer_of_cake @ 400mbps down. I'm looking at various sources of information regarding IRQ balancing for this board, so hopefully with some tweaking I can get it running a little smoother at the top end. :slight_smile:

UPDATE
So I realized BanIP was doing some initial heavy crunching during my speed testing earlier today. This little device is running layer_of_cake on my 400x20mbps DOCSIS connection just fine. I'm getting 420x23mbps without bufferbloat, just like I was with my 4 vCPU @ 3.4ghz VM. Obviously, my overall CPU utilization is higher on this device, but it's keeping up and has room to spare.

I switched over to the performance governor for a bit and ran some speed tests. I saw no discernible difference, other than temps being higher (~65c vs ~55c while not under 400mbps load). I am going to play around with that some more though. Also, as others have noted, it does seem that CPU0 takes the brunt of the load and I'm sure that's causing some bottlenecks.

UPDATE 2

............................................................
 Download: 411.14 Mbps
  Latency: [in msec, 61 pings, 0.00% packet loss]
      Min:  19.265
    10pct:  21.325
   Median:  23.552
      Avg:  23.917
    90pct:  26.471
      Max:  32.481
 CPU Load: [in % busy (avg +/- std dev) @ avg frequency, 55 samples]
     cpu0:  83.5 +/-  2.4  @ 1512 MHz
     cpu1:  94.6 +/-  1.6  @ 1512 MHz
     cpu2:  38.4 +/-  7.0  @ 1512 MHz
     cpu3:  47.4 +/-  5.6  @ 1512 MHz
 Overhead: [in % used of total CPU available]
  netperf:  19.1
.............................................................
   Upload:  23.24 Mbps
  Latency: [in msec, 61 pings, 0.00% packet loss]
      Min:  19.989
    10pct:  20.885
   Median:  22.832
      Avg:  23.184
    90pct:  25.938
      Max:  29.071
 CPU Load: [in % busy (avg +/- std dev) @ avg frequency, 57 samples]
     cpu0:  13.6 +/-  5.7  @  973 MHz
     cpu1:  34.8 +/-  2.6  @  882 MHz
     cpu2:  11.4 +/-  5.3  @  986 MHz
     cpu3:  13.2 +/-  6.7  @ 1046 MHz
 Overhead: [in % used of total CPU available]
  netperf:   1.2
1 Like

Have you tried this? https://github.com/KaneGreen/R2S-OpenWrt/blob/master/PATCH/irq_optimize.sh

UPDATE:

Date: 24th June 2020

Link: https://github.com/jayanta525/openwrt-nanopi-r2s

  • Repository has been updated with the latest source from OpenWrt master.

  • Added CI/CD example for automated builds with cache enabled

  • LAN: rtl8152 (eth1)

  • WAN: rtl8211E (eth0)

CI/CD Guide with CircleCi:

Link: https://github.com/jayanta525/openwrt-nanopi-r2s/tree/circleci

  1. Fork the on repository

  2. Install CircleCi free plan from GitHub Marketplace

  3. Enable repository in CircleCi Projects

  4. The initial build will be on the default branch with dummy configuration.

  5. Switch to branch "circleci"

  6. Edit feeds.conf to your requirement (specify commit hash values or new feeds)

  7. Edit config.seed file to your requirements (leave the default configuration as is, contains openssl enhancements)

  8. Edit custom script in scripts/ if required.

  9. Commit the changes.

  10. Build in CircleCi will automatically start (if configured correctly, check for orange dot next to commit in GitHub)

  11. Cache is enabled, i.e. toolchains and packages will be cached for faster subsequent compilations (~20mins).

  12. To clear the cache, change $CACHE value in .circleci/config.yml

  13. IRQ_optimize initscript is added, check scrips/adjust-feeds.sh

  14. You'll find the compiled images in artifacts section:

Downloads:

  • ext4 image: link
  • squashfs image (prefered): link
  • config.seed: link

GitHub Releases

Build from sources:

  1. Clone this repository: https://github.com/jayanta525/openwrt-nanopi-r2s.git (git pull won't work as i have force pushed some changes)

  2. Follow these commands:

git clone --depth 1 https://github.com/jayanta525/openwrt-nanopi-r2s.git openwrt/

cd openwrt/

./scripts/feeds update -a

./scripts/feeds install -a

make menuconfig

make -j5 #where 5 is the number of cores in your build system + 1

You'll find the images in openwrt/bin/targets/armv8/

Issues

  • You might come across a kernel config prompt (when verbosity is set), just ignore it with either Y or N, or you can build without verbosity to avoid this prompt.

Commits

TO-DO:

  • Wireless Dongle Drivers
6 Likes

@jayanta525 I am trying to build off your latest rk3328 branch (not doing the CircleCI yet) and run into an issue where the build just hangs fairly early in the process:

make[5]: Leaving directory '/owrt/openwrt-nanopi-r2s/build_dir/target-aarch64_generic_musl/linux-rockchip_armv8/linux-5.4.48'
grep '=[ym]' /owrt/openwrt-nanopi-r2s/build_dir/target-aarch64_generic_musl/linux-rockchip_armv8/linux-5.4.48/.config.set | LC_ALL=C sort | mkhash md5 > /owrt/openwrt-nanopi-r2s/build_dir/target-aarch64_generic_musl/linux-rockchip_armv8/linux-5.4.48/.vermagic
touch /owrt/openwrt-nanopi-r2s/build_dir/target-aarch64_generic_musl/linux-rockchip_armv8/linux-5.4.48/.configured
rm -f /owrt/openwrt-nanopi-r2s/build_dir/target-aarch64_generic_musl/linux-rockchip_armv8/linux-5.4.48/vmlinux /owrt/openwrt-nanopi-r2s/build_dir/target-aarch64_generic_musl/linux-rockchip_armv8/linux-5.4.48/System.map
make -C /owrt/openwrt-nanopi-r2s/build_dir/target-aarch64_generic_musl/linux-rockchip_armv8/linux-5.4.48 KCFLAGS="-ffile-prefix-map=/owrt/openwrt-nanopi-r2s/build_dir/target-aarch64_generic_musl=target-aarch64_generic_musl" HOSTCFLAGS="-O2 -I/owrt/openwrt-nanopi-r2s/staging_dir/host/include  -Wall -Wmissing-prototypes -Wstrict-prototypes" CROSS_COMPILE="aarch64-openwrt-linux-musl-" ARCH="arm64" KBUILD_HAVE_NLS=no KBUILD_BUILD_USER="" KBUILD_BUILD_HOST="" KBUILD_BUILD_TIMESTAMP="Wed Jun 24 15:05:41 2020" KBUILD_BUILD_VERSION="0" HOST_LOADLIBES="-L/owrt/openwrt-nanopi-r2s/staging_dir/host/lib" KBUILD_HOSTLDLIBS="-L/owrt/openwrt-nanopi-r2s/staging_dir/host/lib" CONFIG_SHELL="bash" V=''  cmd_syscalls= KERNELRELEASE=5.4.48 CC="aarch64-openwrt-linux-musl-gcc" modules
make[5]: Entering directory '/owrt/openwrt-nanopi-r2s/build_dir/target-aarch64_generic_musl/linux-rockchip_armv8/linux-5.4.48'
  HOSTCC  scripts/kconfig/conf.o
  HOSTCC  scripts/kconfig/confdata.o
  HOSTCC  scripts/kconfig/expr.o
  LEX     scripts/kconfig/lexer.lex.c
  HOSTCC  scripts/kconfig/preprocess.o
  YACC    scripts/kconfig/parser.tab.[ch]
  HOSTCC  scripts/kconfig/symbol.o
  HOSTCC  scripts/kconfig/lexer.lex.o
  HOSTCC  scripts/kconfig/parser.tab.o
  HOSTLD  scripts/kconfig/conf
scripts/kconfig/conf  --syncconfig Kconfig
net/sched/Kconfig:45: warning: menuconfig statement without prompt
.config:4291:warning: symbol value 'm' invalid for NF_NAT_REDIRECT
.config:6740:warning: override: USB_DWC3_HOST changes choice state
*
* Restart config...
*
*
* ARM64 Accelerated Cryptographic Algorithms
*
ARM64 Accelerated Cryptographic Algorithms (ARM64_CRYPTO) [Y/n/?] y
  SHA-224/SHA-256 digest algorithm for arm64 (CRYPTO_SHA256_ARM64) [N/m/y/?] (NEW)

It's like it's waiting for an interactive response to the CRYPTO_SHA256_ARM64 algorithm and I have no idea why... any ideas?

UPDATE
I ran a make clean, make defconfig, make menuconfig and chose the Rockchip target and profile settings, then a make and run into the same hang with a "clean" config file. Not sure where to go next.

UPDATE 2
Just found this: https://github.com/jayanta525/openwrt-nanopi-r2s/commit/9506ef0ec508b10f2f09762dcf8ded58fa4e39d9#commitcomment-40137810

@jayanta525 Perhaps you'd want to add that verbiage to your last post so others don't chase their tails like I did. :slight_smile:

1 Like

Tomorrow i will get my r2s. Please tell me, i am pretty new to this, how to setup this on my SD card. Is there a manual available?

On my router (C7) i installed and configured openwrt as well and it is working great! Therefore i decided to start using this board

See here for instructions on the SD card prep: Here

I recommend grabbing the EXT4 image from Here to get started. If you're comfortable with building the image, I know the v1.2 code level builds fine.

Sorry, i have a Windows pc. Do i need tot built it or can i simply unpack and copy files to sd?

Have you read through this thread? I think you'll find the info you're looking for in some of the earlier posts. :slight_smile:

As for the SD card + Windows, the image is not just a file where you extract the contents to an SD card. It's literally an image of what the partitions and their contents will be when laid down onto the SD card, so it must be written at the block level of the card. I also use this tool on Mac and there's a Windows version too: https://www.balena.io/etcher/

Thanks, added in the last post: NanoPI R2S is a great OpenWrt device

1 Like

Add this to target/linux/rockchip/armv8/config-5.4 fix waits for an interactive response

CONFIG_CRYPTO_AES_ARM64_CE_BLK=y
CONFIG_CRYPTO_AES_ARM64_CE_CCM=y
CONFIG_CRYPTO_SHA512_ARM64=y
CONFIG_CRYPTO_SHA512_ARM64_CE=y
CONFIG_CRYPTO_SHA3_ARM64=y
CONFIG_CRYPTO_SM3_ARM64_CE=y
CONFIG_CRYPTO_SM4_ARM64_CE=y
CONFIG_CRYPTO_CRCT10DIF_ARM64_CE=y
CONFIG_CRYPTO_AES_ARM64_NEON_BLK=y
CONFIG_CRYPTO_AES_ARM64_BS=y
CONFIG_CRYPTO_ANSI_CPRNG=y
CONFIG_CRYPTO_CMAC=y
CONFIG_CRYPTO_ECB=y
CONFIG_CRYPTO_GHASH_ARM64_CE=y
CONFIG_CRYPTO_MD5=y
CONFIG_CRYPTO_SHA1=y
CONFIG_CRYPTO_SHA1_ARM64_CE=y
CONFIG_CRYPTO_SHA2_ARM64_CE=y
CONFIG_CRYPTO_SHA512=y
CONFIG_CRYPTO_TWOFISH=y
CONFIG_CRYPTO_USER_API_HASH=y
CONFIG_CRYPTO_USER_API_SKCIPHER=y
CONFIG_CRYPTO_DEV_ROCKCHIP=y
CONFIG_SND_SOC_ROCKCHIP=m
CONFIG_SND_SOC_ROCKCHIP_I2S=m
CONFIG_SND_SOC_ROCKCHIP_PDM=m
CONFIG_SND_SOC_ROCKCHIP_SPDIF=m
2 Likes

Update to my previous post on performance Here...

I started to find that after a day or so of operating, my device was having trouble with SQM at my speeds. While on a WebEx meeting yesterday, I ran the speedtest-netperf.sh script on another Linux host (outside of the NanoPi) and noticed some bad side effects. With my SQM + VMware based OpenWrt VM previously, I could easily run any speed test and not suffer any noticeable consequences while on web meetings. Amazing how well SQM can work, really.

Performing the same tests would max out the CPU cores on the NanoPi R2S and about 50% of the time I would literally lose audio and video on my WebEx meeting. SQM stats indicated SQM latency of 8-9 seconds and dropped ACKs between 2,000-3,500 pps over several seconds at a time.

I switched from layer_cake to piece_of_cake and noticed pretty much the same behavior. Given that I have been so spoiled by CAKE as opposed to other qdiscs, I can't imagine switching back to a "lesser" qdisc just to justify sticking with this device at the moment.

I'm not getting rid of it at this point, but I have switched back over to my OpenWrt VM. I am confident this great community will continue to tweak and tune it and it may very well reach a point where I can go back to the NanoPi.

Is anyone else having the same experience at this point?

Well, HTB+fq_codel as in simple.qos will keep latency low even under overload conditions, but it will do so at a potentially significant throughput drop. That said, if your CPUs are busy for seconds on end, SQM is going to be unhappy...

Don't do that then? You can always run a speedtest through the router from another internal host without massively increasing the router's CPU load. Hammering the router with the required netperf/netserver instances is a valid test, but simply not diagnostic for the normal routing performance.

Sorry I was not more clear on this. I do not run the speedtest-netperf.sh script on my router itself for the reasons you mentioned. I run it from another Linux host on my network so as not to consume any OpenWrt resources unnecessarily with the netperf/netserver overhead. The load on the NanoPi that I was seeing was strictly OpenWrt doing routing.

In the interest of transparency, I do have collectd, netdata, and nextdns processes running on OpenWrt, along with the usual suspects (uhttpd, dropbear, banip, and the like). But that is apples-to-apples from a config standpoint between my NanoPi and OpenWrt VM.

I did update my previous post to indicate that the speedtest script was being run outside of the NanoPi box, so as to avoid any questions around that going forward. :slight_smile:

1 Like

Mmmh, so it was just failing hard from trying to shape more bandwidth than cake can actually stomach on that router? Too bad you switched over, because I really would like to see how fq_codel/simple.qos would behave, with and without manual tuning of /usr/lib/sqm/defaults.sh by changing the following line"
[ -z "$SHAPER_BURST_DUR_US" ] && SHAPER_BURST_DUR_US=1000
to say 10000 or 20000. This will increase the burst buffer to 10 or 20ms which both should help to recover a bit of lost throughput without completely destroying real-time performance/ latency under load.

I still have the NanoPi powered up, so I can switch my cables back over to it at some point soon (probably in about two hours) and do those tests for you. :slight_smile:

Something I meant to mention before is that my minimum latency was slightly higher with the NanoPi with the schedutil governor. Even with my OpenWrt VM running on top of a hypervisor, it has lower minimum latency overall. Now, when I switch my NanoPi over to the performance governor, minimum latency is decreased by ~3-5 ms, but it more quickly maxes out the CPU cores, as is logical. This unfortunately leads to the side effects I mentioned before where the CPU was maxed out to the point of SQM choking hard.

Perhaps it's worth noting for the sake of @jayanta525 and @xiaobo's efforts on this, from what I could tell the core that would max out earliest and stay pegged was the one assigned to IRQ 166 (CPU2 in my case), which if I'm not mistaken is the eth1 IRQ:
166: 2024 0 0 18571350 GICv2 99 Level xhci-hcd:usb4

IRQ 28 (eth0) would also get way up there, but only after a few seconds into a speed test when buffers started to really fill up.

1 Like

Thank you very much!