So I opened this thread mostly to gather feedback and assist other new OpenWrt users since it took me a pretty long time to troubleshoot and find the root cause of the issue.
I'm currently using an Ubiquiti UAP-AC-M and a Meraki MR66 as dumb APs in different places flashed with the assorted most recent OpenWrt firmware.
I initially tried to setup the Ubiquiti AP for my home network. Did all my configurations on my main router, switch and AP in order to get it to transmit my wireless vlan in 2 SSIDs under 2.4ghz and 1 SSID under 5ghz.
Now I have around 12 IoT devices and 3 personal consumer devices using the wifi network. Once I set everything up, I turn off my old ISP's wireless router and fire up the Ubiquiti radios. Bit of fine tuning and all the devices immediately appear and authenticate. The results were immediate..
Terrible network quality. Excellent signal strength even with half the driver-rated signal intensity and almost non-existent noise, however I had a ton of timeouts, huge latency spikes even between devices and awful bandwidth. I'm not gonna list all the troubleshooting steps I did yet except if someone asks for more details and to cut it short after hours of troubleshooting and verifying all my equipment was set up properly, I switched to the Meraki MR66 and had the exact terrible results.
Alas, turning on "Allow legacy 802.11b rates" instantly fixed everything, latency back to 2-20ms, full bandwidth, no packet drops, steadfast connection.
Naturally I share this finding here to help and be helped to understand why this happens and how to troubleshoot this as the logs and network info from the CLI didn't help at all. (P.S. All my devices are g/n compatible)