How to minimize risk of ksmbd usage causing Out of Memory (OoM) on device with only 64MB RAM

BorreSanders · February 6, 2025, 12:57pm

Dear reader,
I've successfully installed OpenWRT 24.10.0 on a RAVPower RP-WD009 device. The travel router device has 64MB of RAM, and 16MB of flash. The installation includes ksmbd as lightweight SMB server solution, support for USB bulk storage, exFAT file system, travel mate, and a few LuCI dashboard extensions such as dashboard and battery status. Everything seems kind of functional and healthy at first. Then real life situations kick in.

One of the appealing use-cases is providing WiFi access to SD card contents via ksmbd as SMB server. The server functions, and I'm able to copy small files. However, if I apply copy operations on a big file or stream a high bandwidth video file over SMB, the device crashes and reboots. It appears that the SMB configuration does not respect the memory limits of the device. I've noticed that #25519 suggested ksmbd parameter values (64k for low memory devices such as this one) still result in ksmbd running out of memory (OoM). Reducing the values of parameters smb2 max read = 64K, smb2 max write = 64K, smb2 max trans = 64K from 64k to 1k improves the situation a bit, but does not solve it altogether.

I'm hoping to accomplish a reliable/stable device, with acceptance of some throughput constraint due to respecting memory limitations rather than having a potentially crashing device for any SMB file transfer. I'm no expert on ksmbd and am looking for suggestions on reducing the memory footprint of the service in favor of device stability. Any configuration suggestions that can help secure device stability would be greatly appreciated.

Thanks in advance for your support!

romanovj · February 6, 2025, 1:11pm

Memory usage spike on server during copying 128 MB file (according to htop).

`smb2 max credits = 16` +2MB

`smb2 max credits = 64` +10MB

`smb2 max credits = 512` +44MB

`smb2 max credits = 1024` +100MB

github.com/namjaejeon/ksmbd

router oom on write even with cache trans buffers = no

opened 07:38PM - 25 Dec 24 UTC

closed 11:43PM - 30 Dec 24 UTC

romanovj

ksmbd.conf ``` [global] netbios name = router server string …= Ksmbd on OpenWrt workgroup = WORKGROUP interfaces = bind interfaces only = yes ipc timeout = 20 deadtime = 15 map to guest = Bad User smb2 max read = 64K smb2 max write = 64K smb2 max trans = 64K cache read buffers = no cache trans buffers = no ######### Dynamic written config options ######### [usb] path = /mnt/sdb force user = root force group = root create mask = 0666 directory mask = 0777 read only = no guest ok = yes ``` ksmbd: next 6a50a6c0b123ed8bb6c5c9d19d434de5226cbf39 ksmbt-tools: 3.5.3 kernel: 5.15.167 device: openwrt router, arm64 with 256MB RAM No problems only when mounting cifs with `-o rw,cache=none` vs default `cache=strict`.

slh · February 6, 2025, 1:11pm

CMB/ CIFS is inherently memory hungry, especially with large files. While ksmbd is already an improvement in this regard over the traditional samba dæmon, file servers still need fast I/O and a lot of RAM (personally I wouldn't even try it with less than 512 MB RAM for long term usage). This is very likely beyond the ability of your device (be aware that openwrt >=25.xy.0 will state minimum system requirements of 16/128).

BorreSanders · February 6, 2025, 1:41pm

I was hoping for some missed parameter that I still could tune. Thanks for this suggestion! Adding smb2 max credits = 16 seems to improve the stability situation while copying large files dramatically. I'll do some experimenting, and post my findings later.

BorreSanders · February 6, 2025, 1:43pm

I know I'm pushing the limits, yet it's impressive (and fun) what this affordable travel device is capable of with a bit of parameter tuning. Thanks for thinking along, and the heads up warning.

slh · February 6, 2025, 1:47pm

Personally I was cured from using my router(s) as file servers (even without samba) early on in the bcm47xx/ ath79 days, as performance just too abysmal (back then, 1.5-2 MByte/s…).

BorreSanders · February 6, 2025, 5:26pm

Did a few experiments with playback of video files and copying files. For findings see below.
Conclusions:

adding smb2 max credits = 4 seems to yield the best device stability, and a workable but not perfect situation.
@slh has a valid point on the devices' memory and CPU resources being very constrained for the serving the purpose of also being a robust SMB server.

Explanation:

With value set to 4 I experienced pretty good file copying and video playback performance for a single (SMB) user. Transfer speeds were up to 11MB/s but fluctuating, and based on mtk-sd driver related discussion here I think max transfer speed may be constrained by the SD device/driver involved.
With value set to 4 I was able to successfully copy various 8~10GB files, which I used as load/stress test that the device could not pass before. Value >4 implies higher risk of OoM during SMB operations, and <4 (e.g. 2) implies that SMB becomes unusable.
With value set to 4 device stability during SMB use remains far from perfect, and the device still crashed due to OoM and rebooted during some of the stress tests. However, the configuration is pretty workable for real life situations with copy operations or low data rate video streams (not high data rate 4k streams).
To reduce risk of OoM, it helps to not run any parallel operations on the device. For example, logout from LuCI before copying a large file over SMB, and refrain from using the MiFi connected client device(s) for browsing the internet while copying a large file.

Note:
There are a few packages that I could have uninstalled (luci-mod-battstatus, luci-mod-dashboard, travelmate, luci-app-travelmate) to free up some more resources, but I didn't want to give in on usability that much.

Hope this report back helps a few others on low resource devices with ksmbd.
Credits to @romanovj and @slh for the great hints that made a difference!

romanovj · February 6, 2025, 7:06pm

You can try more tweaks, also install zram

sysctl -w vm.dirty_background_ratio=2
sysctl -w vm.dirty_ratio=5
sysctl -w vm.dirty_expire_centisecs=500
sysctl -w vm.dirty_writeback_centisecs=200
sysctl -w vm.vfs_cache_pressure=1000

BorreSanders · February 6, 2025, 9:50pm

Install of kmod-zram went OK. Tried zram-swap next, assuming that is the right thing to do:

Not sure what to do about this. Any advice or reference to applicable documentation? Or is this expressing a true limitation of the standard kernel for this device, and can the zram scenario just not be applied due to lack of virtual memory support enablement in this config? I think so, and I'm not inclined to customize the community supported kernel configuration for the device. Any thoughts on that, as well as additional - less intrusive - memory usage tweak options are still welcome.

romanovj · February 7, 2025, 11:44am

@BorreSanders
you can check

lsmod | grep zram

modprobe zram

Also, when mounting cifs on linux you can add cache=none to the options. (I don't know how to do it on Windows).

BorreSanders · February 7, 2025, 12:54pm

Sure, see this screenshot for results:

So modprobe yields nothing.

frollic · February 7, 2025, 12:55pm

that's fine ...

try dmesg after the modprobe.

BorreSanders · February 7, 2025, 12:58pm

romanovj · February 7, 2025, 1:43pm

ls /proc/swaps

BorreSanders · February 7, 2025, 2:07pm

romanovj · February 7, 2025, 2:34pm

that's because of small_flash https://github.com/openwrt/openwrt/commit/fd46680966e9b9bc282b665282e2e6a22a27c26e

so no zram for you!

BorreSanders · February 7, 2025, 3:19pm

Thanks for the verification support! I think that concludes the zram experiment.

system · February 17, 2025, 3:19pm

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.