I had a time to play with it a bit more, and down to single reproducible change - VLANs.
- Configure Dumb AP, IPv4 only, dnmasq and firewall disabled. Works.
- Enable VLAN, add single VLAN1=U*, change Interface to br-vlan.1
- If encryption is enabled I can't connect to WiFi. If encryption is disabled - it works.