It is worth mentioning that:
- closed source != insecure
- open source != secure
From a security standpoint, the benefit that open source software has over closed source is that it can be used freely (thus more potential users of a platform) and that the source code can be reviewed by everyone. Between these two aspects, there are more opportunities to find and patch security issues, and it is often done with a reasonable level of with transparency. This doesn't inherently mean that all open source software has indeed been tested, reviewed, and hardened, though, and it also doesn't mean that all security related bugs/vulnerabilities are known to the developers to be resolved.
That said, OpenWrt is considered to be reasonably well tested, audited, secured and hardened (at least in the context of home and small business use).
Meanwhile, it might surprise you to know that the Unifi APs have a codebase that was forked from LEDE (17.01), which was a branch of OpenWrt (between 15.05 and 18.06).
Downstairs-BZ.v4.3.20# cat openwrt_release
DISTRIB_DESCRIPTION='LEDE Reboot 17.01.6 r3979-2252731af4'
DISTRIB_TAINTS='no-all mklibs busybox'
Now, granted, this is a VERY old version and considered to be vulnerable to various security flaws -- it is advisable to avoid installing LEDE (official release) onto any devices that will be connected to the internet based on these issues. However, there are three things that make this less of a concern in the Unifi context:
- The APs running the Unifi stack are not designed to be directly exposed to the internet - they sit behind a router/firewall and purely serve as access points.
- Ubiquiti does release regular updates to the APs which include security patches, bug fixes, and feature additions (although it is very fair to question the quality control of these releases given the last 1+ years of updates causing other types of network issues such as the major DHCP bugs and such).
- The Unifi stack includes some customized security measures in terms of the provisioning and management process, ssh key handling, elimination of LuCI/web interface and default disabling of ssh access after adoption, and several other elements.
Consider that Unifi is aimed at and used by high end home users, small businesses, educational institutions and small-scale (or lower end) enterprise environments. Therefore, it is important that Ubiquiti stays on top of the security vulnerabilities. That's not to say that they don't screw up -- they have had several epic events in the last year or two, but most of those have been related to their cloud services, not local devices.
At the same time, consider your threat models (i.e. the expertise/persistence of attackers, the environment (home vs business), the modes of access (physical, proximity, connected to the same network, over the internet, etc.), the value of your network as a target (a major political campaign would be much higher value than a small town automotive garage), and the consequences of a breach.
Meanwhile, don't assume that the standard Unifi stack is lacking up-to-date security measures and patches. Likewise, don't assume that running OpenWrt immediately means you've got better security -- a flaw can be discovered at any time and/or you could simply misconfigure your device with respect to the various security considerations.
None of this is to say anything negative about OpenWrt -- I'm on this board because I love the platform and I believe that it has a good track record on security (i.e. patched reasonably quickly when issues are discovered, well designed architecture, etc.). If you want to go with OpenWrt, awesome -- I think you'll be really happy with it, and this forum is here to help you learn and troubleshoot.