Samba 4.x package support thread

Community Builds, Projects & Packages
225

I tried your samba4 version now and it gave me the same poor read performance. I tested the device with fio, which gave me about 80 MB/s read performance, if this is true it is not the device it self that causes the problem.

fio --rw=read --name=test --size=1000M --filename=/dev/sda
test: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
fio-3.13
Starting 1 process
Jobs: 1 (f=1): [R(1)][100.0%][r=79.4MiB/s][r=20.3k IOPS][eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=3435: Thu Feb 28 16:57:26 2019
  read: IOPS=20.6k, BW=80.4MiB/s (84.3MB/s)(1000MiB/12432msec)
    clat (usec): min=8, max=13177, avg=24.56, stdev=76.31
     lat (usec): min=10, max=13181, avg=28.69, stdev=76.95
    clat percentiles (usec):
     |  1.00th=[   10],  5.00th=[   13], 10.00th=[   18], 20.00th=[   18],
     | 30.00th=[   18], 40.00th=[   19], 50.00th=[   19], 60.00th=[   19],
     | 70.00th=[   19], 80.00th=[   20], 90.00th=[   21], 95.00th=[   22],
     | 99.00th=[  273], 99.50th=[  297], 99.90th=[  437], 99.95th=[  519],
     | 99.99th=[ 2147]
   bw (  KiB/s): min=68798, max=94064, per=99.92%, avg=82301.79, stdev=4312.91, samples=24
   iops        : min=17199, max=23516, avg=20575.33, stdev=1078.30, samples=24
  lat (usec)   : 10=2.75%, 20=86.46%, 50=7.79%, 100=0.51%, 250=0.87%
  lat (usec)   : 500=1.54%, 750=0.04%, 1000=0.01%
  lat (msec)   : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%
  cpu          : usr=39.54%, sys=57.51%, ctx=1244, majf=0, minf=25
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=256000,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
   READ: bw=80.4MiB/s (84.3MB/s), 80.4MiB/s-80.4MiB/s (84.3MB/s-84.3MB/s), io=1000MiB (1049MB), run=12432-12432msec

Disk stats (read/write):
  sda: ios=3984/0, merge=0/0, ticks=4830/0, in_queue=4820, util=37.28%

I am going to test it with btrfs and etx2 now to see wether it has anything to to with the filesystem.
Any other suggestions?

226

btrfs and ext2 behave in the same odd way. On a wired network I get about 60-80 MB/s write speed, but only 15-30 MB/s read speed.
I assume it has to to with samba, any ideas what I could further check to finally improve the reading speed?

227

Unfortunately i don't really know what you could do, since the config template is pretty much sambas default settings and i never had a report of high write, but low read speeds. The only thing that stands out is that you are on macOS, so as far as i know macOS will connect via smb2.1 instead of smb3. Maybe you can try from a windows/linux machine to verify the slow read speeds on a smb3 capable OS? You could also try a different hdd/sdd just to verify its not the hardware.

PS: How do you actually measure the read speeds?

228

Can you help the file can be seen issue?

Sat Mar  2 08:22:27 2019 daemon.err nmbd[19262]:   daemon_ready: STATUS=daemon 'nmbd' finished starting up and ready to serve connections
Sat Mar  2 08:22:27 2019 daemon.err smbd[19261]: [2019/03/02 08:22:27.664742,  0] ../lib/util/become_daemon.c:138(daemon_ready)
Sat Mar  2 08:22:27 2019 daemon.err smbd[19261]:   daemon_ready: STATUS=daemon 'smbd' finished starting up and ready to serve connections
Sat Mar  2 08:22:47 2019 daemon.err smbd[19316]: [2019/03/02 08:22:47.379929,  0] ../lib/util/charset/codepoints.c:16656(get_conv_handle)
Sat Mar  2 08:22:47 2019 daemon.err smbd[19316]:   dos charset 'CP850' unavailable - using ASCII
Sat Mar  2 08:22:47 2019 daemon.err smbd[19316]: [2019/03/02 08:22:47.395115,  0] ../lib/util/charset/codepoints.c:16656(get_conv_handle)
Sat Mar  2 08:22:47 2019 daemon.err smbd[19316]:   dos charset 'CP850' unavailable - using ASCII
Sat Mar  2 08:22:50 2019 daemon.err nmbd[19262]: [2019/03/02 08:22:50.640436,  0] ../source3/nmbd/nmbd_become_lmb.c:397(become_local_master_stage2)
Sat Mar  2 08:22:50 2019 daemon.err nmbd[19262]:   *****

root@OpenWrt:~# testparm -s -v | grep charset
rlimit_max: increasing rlimit_max (1024) to minimum Windows limit (16384)
WARNING: The "null passwords" option is deprecated
Registered MSG_REQ_POOL_USAGE
Registered MSG_REQ_DMALLOC_MARK and LOG_CHANGED
Load smb config files from /etc/samba/smb.conf
rlimit_max: increasing rlimit_max (1024) to minimum Windows limit (16384)
WARNING: The "null passwords" option is deprecated
Processing section "[sda]"
Loaded services file OK.
WARNING: socket options = TCP_NODELAY IPTOS_LOWDELAY SO_RCVBUF=65536 SO_SNDBUF=65536
This warning is printed because you set one of the
following options: SO_SNDBUF, SO_RCVBUF, SO_SNDLOWAT,
SO_RCVLOWAT
Modern server operating systems are tuned for
high network performance in the majority of situations;
when you set 'socket options' you are overriding those
settings.
Linux in particular has an auto-tuning mechanism for
buffer sizes (SO_SNDBUF, SO_RCVBUF) that will be
disabled if you specify a socket buffer size. This can
potentially cripple your TCP/IP stack.

Getting the 'socket options' correct can make a big
difference to your performance, but getting them wrong
can degrade it by just as much. As with any other low
level setting, if you must make changes to it, make
 small changes and test the effect before making any
large changes.

Server role: ROLE_STANDALONE

	dos charset = CP850
	unix charset = UTF-8

The samba configuration:

[global]
	netbios name = |NAME| 
	interfaces = |INTERFACES|
	server string = |DESCRIPTION|
	#unix charset = |CHARSET|
        unix charset = UTF-8
        dos charset = CP850
	workgroup = |WORKGROUP|

	## This global parameter allows the Samba admin to limit what interfaces on a machine will serve SMB requests.
	bind interfaces only = yes
	
	## time for inactive connections to-be closed in minutes
	deadtime = 15
	
	## disable core dumps
	enable core files = no
	
	## set security (auto, user, domain, ads)
	security = user
	
	## This parameter controls whether a remote client is allowed or required to use SMB encryption.
	## It has different effects depending on whether the connection uses SMB1 or SMB2 and newer:
	## If the connection uses SMB1, then this option controls the use of a Samba-specific extension to the SMB protocol introduced in Samba 3.2 that makes use of the Unix extensions.
	## If the connection uses SMB2 or newer, then this option controls the use of the SMB-level encryption that is supported in SMB version 3.0 and above and available in Windows 8 and newer. 
	##(default/auto,desired,required,off)
	#smb encrypt = default
	
	## set guest user
	guest account = root

	## set invalid users
	#invalid users = root
	
	## map unknow users to guest
	map to guest = Bad User

	## allow client access to accounts that have null passwords. 
	null passwords = yes
	
	## The old plaintext passdb backend. Some Samba features will not work if this passdb backend is used. (NOTE: enabled for size reasons)
	## (tdbsam,smbpasswd,ldapsam)
	passdb backend = smbpasswd
	
	## Set location of smbpasswd ('smbd -b' will show default compiled location)
	#smb passwd file = /etc/samba/smbpasswd 
	
	## LAN/WAN options (IPTOS_LOWDELAY TCP_NODELAY) WAN (IPTOS_THROUGHPUT)
	socket options = TCP_NODELAY IPTOS_LOWDELAY SO_RCVBUF=65536 SO_SNDBUF=65536
	
	## lower CPU useage if supported
	use sendfile = yes
	
	## samba will behave as previous versions of Samba would and will fail the lock request immediately if the lock range cannot be obtained.
	#blocking locks = No
	
	## disable loading of all printcap printers by default (iprint, cups, lpstat)
	load printers = No
	printcap name = /dev/null

	## Enabling this parameter will disable Samba's support for the SPOOLSS set of MS-RPC's.
	disable spoolss = yes
	
	## This parameters controls how printer status information is interpreted on your system.
	## (BSD, AIX, LPRNG, PLP, SYSV, HPUX, QNX, SOFTQ)
	printing = bsd
	
	## Disable that nmbd is acting as a WINS server for unknow netbios names
	#dns proxy = No
	
	## win/unix user mapping backend
	#idmap config * : backend = tdb
	
	## Allows the server name that is advertised through MDNS to be set to the hostname rather than the Samba NETBIOS name.
	## This allows an administrator to make Samba registered MDNS records match the case of the hostname rather than being in all capitals.
	## (netbios, mdns)
	mdns name = mdns
	
	## Clients that only support netbios won't be able to see your samba server when netbios support is disabled.
	#disable netbios = Yes
	
	## Setting this value to no will cause nmbd never to become a local master browser.
	#local master = no

	## (auto, yes) If this is set to yes, on startup, nmbd will force an election, and it will have a slight advantage in winning the election. It is recommended that this parameter is used in conjunction with domain master = yes, so that nmbd can guarantee becoming a domain master. 
	#preferred master = yes

	## (445 139) Specifies which ports the server should listen on for SMB traffic.
	## 139 is netbios/nmbd
	#smb ports = 445 139
	
	## This is a list of files and directories that are neither visible nor accessible.
	## Each entry in the list must be separated by a '/', which allows spaces to be included in the entry. '*' and '?' can be used to specify multiple files or directories as in DOS wildcards.
	veto files = /Thumbs.db/.DS_Store/._.DS_Store/.apdisk/
	
	## If a directory that is to be deleted contains nothing but veto files this deletion will fail unless you also set the delete veto files parameter to yes.
	delete veto files = yes
	
################ Filesystem and creation rules ################
	## reported filesystem type (NTFS,Samba,FAT)
	#fstype = FAT
	
	## Allows a user who has write access to the file (by whatever means, including an ACL permission) to modify the permissions (including ACL) on it.
	dos filemode = Yes
	
	## file/dir creating rules
	#create mask = 0666
	#directory mask = 0777
	#force group = root
	#force user = root
	#inherit owner = windows and unix
################################################################

Also I have compile the kmod-nls-CP850, and compile with GLIBC.

229

Just remove SO_RCVBUF=65536 SO_SNDBUF=65536 and dos charset = CP850, what are you trying to accomplish with those options?

230

dos charset = CP850 this is added after the samba report
dos charset 'CP850' unavailable - using ASCII

So remove these options still no help.

This error only reported for some files, maybe this files with different charset.

231

Do you get those errors only with dos filemode = Yes, what happens if you disable it? I have never seen those errors before, so can only guess here.

232

Okay, Thanks for your reply! I will try it with a windows machine as soon as I have one.
regarding the way I measure the speed. I just use a tool that throws out the transfer speed and I am copying a several gigabyte sized file.

233

I have another problem regarding Time Machine. It works for the first backup, but afterwards macOS is complaining to not be able to find the disk for the subsequent backups.
Anybody knows this problem, plus a solution?
I can still see the Time Machine share and re-add it to Time Machine, but than the same thing happens again, first Backup ok and the following wont work.

here is the relevant error messages from macOS:

Mount URL: request 291527C2-8036-45DB-9DE4-5C45C3004288 from backupd (11894)
URL = smb://GUEST:********@OpenWrt._smb._tcp.local./backups
server smb://OpenWrt._smb._tcp.local./backups
server smb://GUEST:********@OpenWrt._smb._tcp.local./backups
URL  smb://GUEST:********@OpenWrt._smb._tcp.local./backups
Mount point = /Volumes/com.apple.TimeMachine.backups-794729AD-B3D9-4FC9-B6EF-8EE8CD4C868D
smb_mount: mount failed to OpenWrt._smb._tcp.local./backups, syserr = Invalid argument
NAConnectToServerSync failed with error: 22 (Invalid argument) for url: smb://GUEST:@OpenWrt._smb._tcp.local./backups
Backup failed with error 19: The backup disk could not be resolved, or there was a problem mounting it.
234

Am having exactly the same issue. Haven’t found a solution for it either. Deleting all files will allow it to make an initial backup again, but only once. Am wondering if it has something todo with the quota system

235

yes its the quota system, when turned of its working!

236

worthless option though as it will then never purge older backup files.

237

Is this a macOS side option or something i need to change on the samba side?

238

I have already tried this before report this issue, but still thanks for your help.

239

It seems related to iconv, I have try to remove without-iconv and selected the libiconv and libcharset, seems not work.

Checking for library iconv : no
Checking for library iconv : not found
Checking for iconv_open : ok
Checking errno of iconv for illegal multibyte sequence : ok

Don't know how to enable it.

240

I need to re-verify the waf-cross answer files, thats where the conversion settings come from.

Checking if can we convert from CP850 to UCS-2LE: OK
Checking if can we convert from UTF-8 to UCS-2LE: OK

Have not validated those in a while, so maybe something broke there. You can try use my feed version, which has qemu support to test those.

241

Seems to be a bug on samba side. Fruit adds support for a maximum size of the timemachine share. The mac will keep filling up the share until it runs out of space and will then start to delete the oldest file.

My previous homebrew build with a manually configured share worked ok, but the version where you added support for a maximum size in the gui, doesn’t work.

My assumption is that this is because Samba needs to enumerate the backup files and log their size, and it needs filesystem permission to do that. Now that Samba is no longer running as root it might have insufficient priviliges to read the files that are created with very strict permissions.

As soon as i have some time, i’ll test this assumption, by running Samba as root again.

Edit: wrong assumption. Samba still runs as root...
I mixed up Mariadb and Samba in my foggy mind.

@Andy2244: Any suggestion how we can turn up debugging to the max and figure out on the Samba side what is happening?
Smb.conf seems ok, this is not a simple config error

242

Are you using my latest version, with the tmsize patch?

243

Guess not. Still running code from nov :confounded:
Will build a new image and check if that helps.

244 
Checking uname sysname type: "Linux"
Checking uname machine type: "armv7l"
Checking uname release type: "4.15.0-45-generic"
Checking uname version type: "#48~16.04.1-Ubuntu SMP Tue Jan 29 18:03:48 UTC 2019"
Checking simple C program: "hello world"
rpath library support: OK
-Wl,--version-script support: OK
Checking getconf LFS_CFLAGS: NO
Checking for large file support without additional flags: NO
Checking for -D_FILE_OFFSET_BITS=64: OK
Checking correct behavior of strtoll: NO
Checking for working strptime: OK
Checking for C99 vsnprintf: "1"
Checking for HAVE_SHARED_MMAP: OK
Checking for HAVE_MREMAP: OK
Checking for HAVE_INCOHERENT_MMAP: NO
Checking for HAVE_SECURE_MKSTEMP: OK
Checking for HAVE_IFACE_GETIFADDRS: OK
Checking value of NSIG: "65"
Checking value of _NSIG: "65"
Checking value of SIGRTMAX: "64"
Checking value of SIGRTMIN: "34"
Checking for a 64-bit host to support lmdb: NO
Checking errno of iconv for illegal multibyte sequence: (255, "")
Checking for kernel change notify support: OK
Checking for Linux kernel oplocks: OK
Checking for kernel share modes: OK
Checking whether POSIX capabilities are available: OK
Checking if can we convert from CP850 to UCS-2LE: (255, "")
Checking if can we convert from IBM850 to UCS-2LE: (255, "")
Checking if can we convert from UTF-8 to UCS-2LE: OK
vfs_fileid checking for statfs() and struct statfs.f_fsid: OK
Checking whether we can use Linux thread-specific credentials with 32-bit system calls: OK
Checking whether fcntl locking is available: OK
Checking whether fcntl lock supports open file description locks: NO
Checking for the maximum value of the 'time_t' type: NO
Checking whether the realpath function allows a NULL argument: OK
Checking for ftruncate extend: OK
getcwd takes a NULL argument: OK