Samba cross-subnet browse list collation not working

Given that the upstream Samba folks don't support 3.6 anymore, I'm not optimistic about getting this fixed, but I'll put this up here anyway: I'm trying to implement cross-subnet network browsing using Samba, and browse list sync is not happening. It's a cross-subnet Windows Workgroup (called "WORKGROUP") with a WINS server; there's no Windows Domain or Primary Domain Controller. This is on OpenWRT 18.06.1 on 3 LInksys WRT1200AC routers, and replaces 3 tired old Netgear WNR3500L routers that were running DD-WRT and doing the same thing (with Samba version 2).

One router is configured as the OpenVPN (routed, not bridged) server and also as the WINS server and the Samba Domain Master Browser (DMB) for the workgroup. Two other routers are OpenVPN clients and serve as the Local Master Browsers (LMBs) on their subnets (verified using nbtstat), but are not allowed to be the DMB and they are told the IP address of the WINS server in their configurations. The DMB is also its subnet's LMB and shows as the DMB in nbtstat. According to the Samba doc on version 3.2, when a cross-subnet workgroup is configured this way, the DMB is supposed to automatically collect and collate browse lists from the remote LMBs and send a complete browse list back to the remote LMBs.

Direct quote from the Samba3 HOWTO:
" Where a WINS server is used, the DMB registers its IP address with the WINS server using the name of the domain and the NetBIOS name type 1B (e.g., DOMAIN<1B>). All LMBs register their IP addresses with the WINS server, also with the name of the domain and the NetBIOS name type of 1D. The 1B name is unique to one server within the domain security context, and only one 1D name is registered for each network segment. Machines that have registered the 1D name will be authoritive browse list maintainers for the network segment they are on. The DMB is responsible for synchronizing the browse lists it obtains from the LMBs."

So this means I should expect to find some "WORKGROUP<1D> .." entries with the IP addresses for the remote LMBs in /var/lock/wins.dat on the router hosting the WINS server, correct? But no such entries ever occur. Furthermore, the browse lists on each router, at /var/lock/browse.dat, are not empty and contain unique entries for other client computers. So these lists are being created locally, just not synchronized.

I have also used the "remote announce" and "remote browse sync" options in an attempt to force synchronization, but so far all I have seen is the remote routers appear in the browse lists locally; so the "remote announce" appears to work, but there is still no browse list sync occurring. BTW, for these options the arguments have been the IP addresses of every other router remote to the local one.

If anyone has some ideas about how to make this work, I'd very much appreciate it. I will add here that I also serve the /etc and /var directories of each router on the LAN and over the VPN to make my job easier, and these "Samba server" resources do appear in /var/lock/wins.dat on the WINS server. This is confirmation that the VPN tunnel is allowing registration traffic to the WINS server. Also, I had to override the smb.conf.template "interfaces = |INTERFACES|" entry by commenting it out and adding in the line "interfaces = lo br-lan tun0" so that the VPN tunnels were included in the interface list for Samba. This is reflected in /var/etc/smb.conf on each router.

I'll also generally add here that the VPN tunnel traffic flows freely to and from the bridged LAN (br-lan) interface at each router, except for DHCP traffic being blocked. Remote desktop connections (RDP and VNC) and direct share connections work just fine. The only thing that appears to be busted is the browse list collation.

I would like to ask the developers one question: I can see that the procd init script command starts up nmbd with no file options (just '-F'). I take it this means that /var/etc/smb.conf is the default location for the smb.conf file that is baked into the code. Is this correct? If not, the init script might need some work.

One idea I may pursue is creating an lmhosts file on the WINS/DMB router and including the "WORKGROUP<1D>..." entries there to see if that has any positive effect. But Microsoft's documentation on lmhost file creation is very spotty, and I'm not sure that these '1D' entries without corresponding name entries would work. Any lmhosts experts out there?

Actually /etc/samba/smb.conf is the default location, can be checked via: smbd -b | grep conf. The init scripts adds some dynamic entries to the base conf and links it via /var/etc/smb.conf.

I cant really help, hardly understand what the problem is. You can try samba4 (snapshots), but netbios (nmbd) is not build anymore by default, but can be added as build option.

PS: I also currently try to get a new kernel module based smb3 server added, as a alternative to samba3/4. Still need's some work before i can add the PR, but since its much lighter and way easier to build, maintain i might add a backport PR from 18.06.1.

1 Like

Andy, thanks for the reply. Other than procd monitoring for changes to /var/etc/smb.conf after the init script creates it, there seems to be no linkup between that file and the executables at all. I don't see explicitly how smbd or nmbd are ever told that /var/etc/smb.conf is the config file to use at the spots they are run (and would very much appreciate an explanation of how that works), and ps shows no config file option passed to the running instance, just the '-F' to run it in the foreground. So this gives me a few other things to look into when I can. These routers are deployed in a network that runs a small business, so I'm cautious about working on them when the business is running.

Beyond the couple of things to do above, I will be interested in replacement packages for Samba36 as well. I would like to see a backport of Samba4x for OpenWRT 18.06.1 and I wonder why it hasn't happened yet since the upstream folks abandoned Samba36 some time ago. As long as your kernel module includes nbmd (WINS) support and is configurable via LuCI, I'll be interested, since nmbd is critical for the network while smbd on the routers is only useful for me. (The business runs separate NAS computers for storage.) What function are you planning to include in the kernel module?

In the init file this lines does the linking:
[ -L /etc/samba/smb.conf ] || ln -nsf /var/etc/smb.conf /etc/samba/smb.conf

I did not backport samba4, because it just was stabilized a few months ago and i still have a few things on the to-do list aka hotplug support. It will also kinda break the old setups, since smbv1/netbios is not supported anymore out of the box. In most cases samba3.6 still does a decent job, so i kept samba4 in snapshots for now. I will probably backport the kernel module before samba4 if all goes well, since its more usefully as a samba3.6 replacement.

PS: I just adapt the kernel project to build/run on openwrt, so have not tested if netbios is even fully supported. Will take some time to get this in shape and tested.

1 Like

Ah, the dreaded symlink. I should have spotted that. So looking at ps output alone is misleading. Unfortunately, if the Samba guys have also defeatured NetBIOS in Samba4, then it won't help me either -- operation of the network relies upon the NetBIOS over TCP/IP protocol (DHCP, i.e. dnsmasq, configures all devices as hybrid nodes, which in turn makes possible directed UDP packets in lieu of broadcasts), a single WINS server to do network-wide name resolution, and a pre-configured agreement among the Local Master Browsers as to which is the Domain Master Browser. NetBIOS undergirds the whole thing; take it out and the whole thing collapses.

I should have been more clear, what i meant is that Windows10 disables smbv1 by default and the related explorer service to find shares. Windows 10 now uses wsd and lldp as default way, while smbv1 can still be enabled manually. Netbios still works in Windows 10 and samba4, just some defaults changed. In response i created the samba4 package to not build the nmbd anymore, since normal discovery wont work, without enabling smbv1. So instead i added the wsdd2 package to take care of Windows 10 clients and for macOS, Linux i made sure the build is AVAHI (mdns) compatible.

So if you build samba4 with netbios support and manually setup your configuration anyway, it should still work.

1 Like

Andy, we know that TPTB at Microsoft are readying a tombstone for SMBv1/NetBIOS/WINS, but were hoping to see this router hardware and firmware upgrade work as a drop-in replacement while getting us over to platforms that can be migrated to the WSD and LLDP way of doing things, something not possible with the old hardware. I will still do a couple of more investigative things here, mostly involving rigging up an lmhosts file to live on the WINS server in order to see if I can coax those entries into the /var/lock/wins.dat file manually. I also have an old WNR3500L client router (i.e. remote LMB) with DD-WRT (Samba 2.x) here and I want to see if it will make that registration entry in the WINS database.

But an emerging issue I see with WSD is the low TTL on the packets. Seems to me that this is intended to assure that WSD broadcasts die on LAN segments and don't pass routers, whereas I'd want those packets to cross over 3 routers to span the VPN topology, but be positively dropped at WAN interfaces. Seems to be mostly firewall work, but have you addressed this at all in your wsdd package? And is there LuCI support for configuring this kind of stuff?

Wouldn't it be easier to set up a PC or even a VM with CentOS/Fedora/Debian/Ubuntu and use current version Samba from the repos?

1 Like

@vgaetera: Maybe, as a test. Except for a couple of things: One, the routers are always on and so are ideal candidates to host the LMB/DMB functions as well as the WINS server. And two, while I can make myself just another remote LMB location no matter where I am (and actually, I did that to facilitate this migration, and it worked wonderfully), the router that is the VPN and WINS server and DMB lives physically at a location 350 miles from me, and practically speaking it's not the simplest thing to coordinate this kind of experimental work from my location, especially if it involves deploying more hardware.

Moreover, the folks there are running a business first; this stuff is just a means to their ends. And while they're tech-literate users, they are not developers or engineers and aren't likely to want to participate in having their business network become an experimental development environment... and neither would I, due to the risk of damaging the business if the network were to experience extended downtime as a result of my activities.

On my previous work I set up Samba AD DC @ libvirt+LXC + a bunch of domain member servers/clients including remote department over VPN so I feel some kind of sympathy. =)
However, if Samba is really a critical infrastructure service for business, running it on a router is inappropriate at least. And if you want to avoid accidents virtualization is not just an option, but must be utilized.

I'm following this discussion with some interest. Most of the systems I use are Linux only, and I've never seen anyone actually able to utilize windows network browsing for anything useful outside 5 guys with laptops all trying to share a few files directly off their machines... For big deployments like my wife's work at a university campus, there's just too much stuff you can't find it by browsing. For smaller deployments you're usually centralizing storage to a few servers and you map each one as a drive... I've never really seen network browsing as a thing that anyone actually did. :wink: This is probably my ignorance more than anything.

2 Likes

@vgaetera: Sounds like you were involved in a much larger scale operation than I am. In my case, the business has three locations, two of which involve retail points of sale. The network can't be down during business hours. But AD is not something they have evolved to -- they just recognize the value of the VPN and easy share and display access to remote computers -- so the multi-site Windows Workgroup model suits them well. They do deploy their own Windows computers but they are not Linux-literate to a great degree and I wouldn't ask them to do that just to suit me. BTW, as I mentioned earlier, we're not relying upon Samba in the routers for share access to anything used by the business -- for that they have NAS computers. We're just using Samba for master browsers and WINS, and whether "inappropriate" or not, since the VPN server and clients run on the routers 24/7, it made sense to use Samba on the routers just for the LMB/DMB/WINS as well.

@vgaetera: Hey, did you ever have to look at WINS database to see if expected entries made it in there? I mentioned above that this Samba3-based network is failing to show remote LMBs register in the WINS server with the 'WORKGROUP<1D>' group entry and their IP addresses, and I'm looking to cram those into an lmhosts file. Samba docs I can find say they only support two fields in the lmhosts file (i.e. NetBIOS name and type together as one field and IP address as the other) while Microsoft docs say one can force lmhost entries into WINS by adding a '#PRE' field after the IP address. Ever had to do or try that? That's what I'm contemplating trying. But it will have to wait until the weekend. What I'm curious to see is whether Samba will take the '#PRE' directive and use it constructively and without crashing.

Actually NAS was mapped as a network drive, so there was no need for browsing, similar to the case @dlakelan described above.
However some features required WINS support, so I delegated name resolving to DNS-backend:

wins support = yes
dns proxy = yes
1 Like

Use tcp dump on the gateways....

Try copying the /var/etc/smb.conf to /etc/samba/

Or edit the init.d to specify a conf for nmbd;

-s, --configfile=CONFIGFILE Use alternate configuration file

also try;
-wins proxy
-remote announce

And watch your gateway firewall / tcpdumps... Technically you should only need remote browse sync on all nmbd's.

A last resort would be something like a direct gre p-t-p between the nmb instances....

The WSDD2 service is rather new in openWRT, so far it just works out of the box without any luci-ui options. It checks the smb.cfg file and announces whats listed there. If there is a problem, maybe we can patch it so it works for you, since the code is not this complicated.

1 Like

@anon50098793: Good suggestion on tcpdump. I'll give that a shot. On the others, Andy has me convinced that the config file is getting properly loaded through the symlink (and I've seen behavioral changes reflected when I changed smb.conf.template, too), and I already have "wins proxy" and "remote announce" options in the config. Turns out "remote announce" does seem to work -- I can see the remote LMBs pop up in the /var/lock/browse.dat files after making that change, so it works... sadly, "remote browse sync" just refuses to work.

Would you mind expanding on your comment about only needing remote browse sync on nmbd? As long as it's in the smb.conf, does it matter that smbd sees it? Also, please expand on "direct gre p-t-p" between nmb instances... Do you mean set up a separate point-to-point protocol link just for nmbd?

Yup direct "sub-tunnel"... i think gre or tap... might be most doable....

No smb does not care about those options whatsoever.

remote browse sync most importantly forces nmbd into unicast propagation. adding this parameter is what gets you across subnets assuming routes are in tact, no firewall etc.

as with anything windows, hostnames really matter.... at minimum i suggest adding the names of the remote servers to /etc/hosts

yes, dns can work too in large distributed setups, but it's a little painful to whip up for small systems.... ( hence NETBIOS :wink: )

foreground debug / interactive mode is helpful as are logs...

nmblookup is superhelpful... , especially in determining if it's the network or the server......

nmblookup -S '*'
nmblookup -MRS -d 5 WORKGROUP
as is nbtscan
nbtscan -v -s : 10.2.3.0/24
nmblookup -M -- -

os level would not hurt..... your symptoms could be indicative of BROWSER ELECTION taint.... ( dhcp option to clients helps here )

So armed with those tools, you can pretty much iron it down to;

-packet format / gateway issue
-nmbd compile / config / subsystem issue
-bi directionality / broader network interaction issue....

Also, mocking up two subnets on a stick ( virtualbox ) that sit right next to each other is a good option in simulating all the "levels" one at a time.... quickly and clearly.... if you verify they work on a stick... add a hop and dump the routers traffic.....

https://www.linuxtopia.org/online_books/network_administration_guides/samba_reference_guide/17_NetworkBrowsing_10.html

"Where a WINS server is used, the DMB registers its IP address with the WINS server using the name of the domain and the NetBIOS name type 1B (e.g., DOMAIN<1B>). All LMBs register their IP addresses with the WINS server, also with the name of the domain and the NetBIOS name type of 1D. The 1B name is unique to one server within the domain security context, and only one 1D name is registered for each network segment. Machines that have registered the 1D name will be authoritive browse list maintainers for the network segment they are on. The DMB is responsible for synchronizing the browse lists it obtains from the LMBs."

A good example of how useful logs can be ( maybe add a switch in /etc/init.d.... );
https://ubuntuforums.org/showthread.php?t=1661085

Hey!, where is resolv order??? ( hosts wins dns maybe )

@anon50098793: Good suggestions all... on that last one, you may not have started at the top of this thread (it's getting long), but that bit of documentation is what I'm trying to coax these LMBs and the WINS server into doing -- i.e. I'm looking for those "WORKGROUP<1D> dotted.decimal.ip.address" entries in /var/lock/wins.dat and I never see them happen. So until I see them, the DMB is off the hook for not doing its job of collating browse lists... though it still bothers me that the "remote browse sync" option doesn't force the issue, either.

Probably getting those extra links set up is not going to happen, for reasons I've already mentioned above in the thread in detail, but gist is that this is a production business network environment and the business owners won't go a bridge too far in experimentation if it costs them in time, money, and the risk of the network being down during business hours.

The virtual idea is a good one too, just a mental block for me as I'm an old hardware guy. I actually tried modeling the 3-router network inside of my own home network (with my own router serving the role of upstream gateway) when I first got these and was flat-out not able to get anything working. So instead I configured one remote router to work with the existing (Netgear DD-WRT) server router, and things came together nicely.

Couple questions I'll throw out if anyone can answer quickly: Which OpenWRT 18.06.1 package has nmblookup and nbtscan in it? I tried doing a web search but didn't get back any immediately useful results. Also, I've read enough to know that Samba will take an lmhosts file with the option '-H /dir/lmhosts' ... but is there a separate option to get it to look at the 'hosts' file that wulfy23 is mentioning? This also reminded me of that old Microsoft tool called nblookup, but I think that one was orphaned with XP.

One last question: wulfy23 mentions the logs, but I've understood for awhile that the Samba builds on OpenWRT (and also DD-WRT) were done with logging disabled. Can anyone confirm here that this is still the case? Samba logging -- were it available -- would greatly help here.

@anon50098793: One other thing I failed to mention: I'm certain that the routers are winning their browser elections because I used nbtstat to check them all, and also because I have the DMB's smb.conf set to "oslevel = 255" and the remote LMBs smb.conf set to "oslevel = 254" ... good suggestion, though...