Slow Samba speed with Flint2 over 2.5gbit ethernet with use of SSD

[Just_A_Name]

Hello all,

I have got a Flint2 (openwrt gl-mt6000) router I did mount a Samsung T7 4TB connected to it. The SSD is formatted in EXT4. I am using a wired 2.5gbit Ethernet port connected to an 2.5gbit port on an Ubuntu PC. When I copy a file from the PC to the SAMBA share I only get about 80 MByte/sec for a file of about 7.7. gig. There is a link here which tells I should be able to reach much higher samba speeds when using EXT4:

Flint2 Samba speed

Installing SAMBA

I did use these links to install SAMBA:

Share USB hard-drive with Samba using LuCI

and

Samba

I did try to change some settings with no result

socket options = IPTOS_LOWDELAY TCP_NODELAY SO_RCVBUF=524288 SO_SNDBUF=524288

and

aio read size = 1
aio write size = 1

dmesg

dmesg shows some errors:

root@OpenWrt:/# dmesg | grep "ERROR"
[ 18.999830] xhci-mtk 11200000.usb: ERROR Transfer event for unknown stream ring slot 1 ep 2
[870464.996803] xhci-mtk 11200000.usb: ERROR Transfer event for unknown stream ring slot 1 ep 2

and

root@OpenWrt:/# dmesg | grep "error"
[ 1.155262] GPT: Use GNU Parted to correct GPT errors.

The openwrt wiki show similar errors in the logfile.

I did try some “quirks” before changing the samba conf as described in here: quircks

They do not work out. Maybe a combination of samba configuration and the quircks is needed, I do not know

Network speed:

root@OpenWrt:/# ethtool lan1
Settings for lan1:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
2500baseT/Full
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
2500baseT/Full
Advertised pause frame use: Symmetric Receive-only
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Link partner advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
2500baseT/Full
Link partner advertised pause frame use: No
Link partner advertised auto-negotiation: Yes
Link partner advertised FEC modes: Not reported
Speed: 2500Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 7
Transceiver: external
Auto-negotiation: on
MDI-X: Unknown
Supports Wake-on: d
Wake-on: d
Link detected: yes

iperf shows:

Connecting to host 192.168.0.249, port 5201
[ 5] local 192.168.0.2 port 45190 connected to 192.168.0.249 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 283 MBytes 2.37 Gbits/sec 0 851 KBytes
[ 5] 1.00-2.00 sec 280 MBytes 2.35 Gbits/sec 0 894 KBytes
[ 5] 2.00-3.00 sec 280 MBytes 2.35 Gbits/sec 0 894 KBytes
[ 5] 3.00-4.00 sec 279 MBytes 2.34 Gbits/sec 0 894 KBytes
[ 5] 4.00-5.00 sec 277 MBytes 2.32 Gbits/sec 0 988 KBytes
[ 5] 5.00-6.00 sec 280 MBytes 2.35 Gbits/sec 0 988 KBytes
[ 5] 6.00-7.00 sec 281 MBytes 2.36 Gbits/sec 0 988 KBytes
[ 5] 7.00-8.00 sec 280 MBytes 2.35 Gbits/sec 0 988 KBytes
[ 5] 8.00-9.00 sec 280 MBytes 2.34 Gbits/sec 0 988 KBytes
[ 5] 9.00-10.00 sec 280 MBytes 2.34 Gbits/sec 0 988 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 2.73 GBytes 2.35 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 2.73 GBytes 2.34 Gbits/sec receiver
iperf Done.

SSD info

root@OpenWrt:/# hdparm -tT /dev/sda1


/dev/sda1:
 Timing cached reads:   1670 MB in  2.00 seconds = 834.85 MB/sec
 Timing buffered disk reads: 498 MB in  3.09 seconds = 161.41 MB/sec

root@OpenWrt:/# lsusb -t
/:  Bus 001.Port 001: Dev 001, Class=root_hub, Driver=xhci-mtk/2p, 480M
/:  Bus 002.Port 001: Dev 001, Class=root_hub, Driver=xhci-mtk/1p, 20000M/x2
    |__ Port 001: Dev 002, If 0, Class=[unknown], Driver=uas, 5000M/x2

I wonder if the Class unknown is okay. I think the reads should be higher here.

when I run htop in ssh I see a relative low memory usage, about 100 MB the router has 1 gig or ram.

Can anyone please help me to get to the higher samba speeds? I think it must be possible. Thanx!

[frollic]

Check CPU load during file transfers

[ncompact]

are you sure that your PC's hard disk can read at a speed higher than "80 MByte/sec"

@Just_A_Name

I am using a wired 2.5gbit Ethernet port connected to an 2.5gbit port on an Ubuntu PC. When I copy a file from the PC to the SAMBA share I only get about 80 MByte/sec for a file of about 7.7. gig.

Does the disk you are using have aligned sectors?

Bevor Sie zur Google Suche weitergehen

Have you tried checking performance (without using the network)

example: create file into mount point ssd (/home from the guide you used if followed exactly)?

time dd if=/dev/full of=/home/test1 bs=1M count=100
rm /home/test1

do you want/can you also test "ksmbd" (less expensive than samba):

[OpenWrt Wiki] ksmbd

Bevor Sie zur Google Suche weitergehen

[wilsonyan]

@frollic

Check CPU load during file transfers

this, also check the irq balance

cat /proc/interrupts

Check to see which irq belongs to the usb (xhci) controller and whether its sharing with everything else. If it is you could try allocating it to a different cpu eg…

echo 1 > /proc/irq/[number of xhci controller here]/smp_affinity_list

That command would move it off cpu0 to cpu1.

One last thing, i’m not sure about these new devices that use the Mediatek chipset but older chips like the mt7621 had a special USB3 controller that didn’t have full usb speed (in one direction ?), it was a limitation of the controller of the chip, not saying that’s is what is happening here, but just saying.

[Just_A_Name]

Thank you for the reply, I did check htop using ssh and system load using Luci.

Looking at htop I see the red bars going to about 50% max and back, jumping up and down. The green blue bar CPU percentage never gets above the 0.0% . I think this is strange.

System load in Luci, well I dont know what to make of it, is it 1% of the CPU sounds like too low to me.

[Just_A_Name]

Thank you for the help

The PC I use to test the network speed also uses a SSD, it can reach speeds well above the 80MByte/sec.

root@OpenWrt:/mnt/sda1/users/Test# time dd if=/dev/full of=/mnt/sda1/users/Test/test1  bs=1M count=100
100+0 records in
100+0 records out
real	0m 0.23s
user	0m 0.00s
sys	0m 0.22s

like this? Does this mean it does 100 MB in a kwarter of a second it is about 400MB a second? Or 100/0.23 = 435 MByte a second? That seems fast enough..

[Just_A_Name]

cat /proc/interrupts doesnt change when copying

root@OpenWrt:/mnt/sda1/users/Test# cat /proc/interrupts
           CPU0       CPU1       CPU2       CPU3       
 11:   16818074   23779140   20752799   18976708     GICv3  30 Level     arch_timer
 24:          0          0          0          0   mt-eint   9 Edge      keys
 61:         13          0          0          0   mt-eint  46 Level     mdio-bus:01
 62:         44          0          0          0   mt-eint  47 Level     mdio-bus:07
 81:          6          0          0          0   mt-eint  66 Level     mt7530
116:         13          0          0          0     GICv3 155 Level     ttyS0
120:   53014094          0          0          0     GICv3 229 Level     15100000.ethernet
121:   56458789          0          0          0     GICv3 230 Level     15100000.ethernet
122:          0          0          0          0     GICv3 142 Level     wdt_bark
123:      16953          0          0          0     GICv3 175 Level     11230000.mmc
124:          0          0          3          0    mt7530   0 Edge      mt7530-0:00
125:          0          0          3          0    mt7530   1 Edge      mt7530-0:01
126:          0          0          0          0    mt7530   2 Edge      mt7530-0:02
127:          0          0          0          0    mt7530   3 Edge      mt7530-0:03
128:     619274          0          0          0     GICv3 205 Level     xhci-hcd:usb1
129:          0          0          0          0     GICv3 148 Level     10320000.crypto
130:          0          0          0          0     GICv3 149 Level     10320000.crypto
131:          0          0          0          0     GICv3 150 Level     10320000.crypto
132:          0          0          0          0     GICv3 151 Level     10320000.crypto
133:   97732126          0          0          0     GICv3 245 Level     mt7915e
IPI0:     44932      42885      85274     128907       Rescheduling interrupts
IPI1:  21127669   35356634   75765431   87266966       Function call interrupts
IPI2:         0          0          0          0       CPU stop interrupts
IPI3:         0          0          0          0       CPU stop (for crash dump) interrupts
IPI4:         0          0          0          0       Timer broadcast interrupts
IPI5:         3          0          0          0       IRQ work interrupts
IPI6:         0          0          0          0       CPU wake-up interrupts
Err:          0

[ncompact]

@Just_A_Name

That seems fast enough..

I'd say so.

You can try "ksmbd" to see if it's "samba" (as I hypothesize) or the network (most likely not).

You should get less CPU load (your process 31542 samba 45.2% cpu).

Ksmbd (Samba3/4 alternative, ex cifsd/smbd) package support thread

This is the official ksmbd (ksmbd-server, ksmbd-utils) support thread. It can be found in snapshots and 19.07/20.x. The ksmbd-server is a tiny Samba3/4 alternative, consisting of a kernel module ksmbd.ko and a userspace service usmbd. The server is smb2/3 compatible and works like a Samba4 drop-in replacement. The wsdd2 service will be compatible and a similar luci-app-ksmbd will be available. Keep in mind ksmbd has its own uci config and separate smb.conf paths (/etc/ksmbd/smb.conf). The act…

[Gingernut]

How do you access the drive on the device you are copying the file to?

[Just_A_Name]

The Flint2 router is connected to a Ubuntu pc on the Ubuntu pc I use the caja filebrowser with

smb://192.168.0.2/smb_ssd/Test

and I basically copy past the file from the pc to the samba share

[Just_A_Name]

Today I ran out of time, I can give it a try another time. I still wonder why I do see such a low memory usage in htop, should I make the buffer larger. Maybe

SO_RCVBUF=524288 SO_SNDBUF=524288

should be larger? I havent got a clue how big they should be.

[eduperez]

@Just_A_Name

The green blue bar CPU percentage never gets above the 0.0% .

I see four CPU bars (one for each core), at 42.2%, 31.6%, 33.6%, and 28.3%… which green blue CPU bar does not go above the 0.0%?

@Just_A_Name

System load in Luci, well I dont know what to make of it, is it 1% of the CPU sounds like too low to me.

System load does not works with percentages: a system load of 1.0 is (roughly) equivalent to a 100% CPU usage on one core.

[Just_A_Name]

@eduperez

I see four CPU bars (one for each core), at 42.2%, 31.6%, 33.6%, and 28.3%… which green blue CPU bar does not go above the 0.0%?

I think it is more a row than a bar in htop under the bars please take a look at the screenshot, the text begins with: PID USER PRI …..

@eduperez

System load does not works with percentages: a system load of 1.0 is (roughly) equivalent to a 100% CPU usage on one core.

There are 4 cores so that means on average a core has a load of approximately 25% that seems to correspond to what I see in htop

[Just_A_Name]

When I take a look at htop it looks like there are only 2 lines with the command /user/sbin/smbd -F active at the time. Should they not all be active?

[Gingernut]

Mount the share using fstab instead and test again.

[Just_A_Name]

mount it on the PC using fstab or on the router?

[Gingernut]

Mount it on the PC

[Just_A_Name]

wow … that makes a difference!!

In the first tests I did use Caja as a file browser on the Client Ubuntu PC. In Caja you can type smb://servername to connect to a samba share, like c:\somewhere_in_Windows. I got rid of the Caja file browser for this test. I did use MC (Midnight commander) and did mount the samba share to the client to /mnt/Test/ in the console.

I think the screenshot below is not reliable, 590MB/s looks more like a cache that is getting its data. The number is too high

at the end the speed drops to a more trustworthy number

Luci:

The Ubuntu client connected to the samba share

htop

It looks much better!! when I have some more time I’ll try to play around with the Samba conf numbers. I do not understand the difference between MC and Luci. anyone a idea? Anyway the transfer speed is much better. I’ll call it a day for now. Thank you all!

[bahtsiz_bedevi]

test the disk speed locally (not via samba)

opkg update && opkg install hdparm
hdparm -t --direct /dev/<your disk>