How to access LAN devices over Wireguard client on my OpenWrt Router?

The theory is correct (mostly). The problem is that the actual fields don't do anything. You need to set it in the dnsmasq configuration, instead. And you can actually use any public DNS or your VPS... there is one major caveat here... if the WG peer config to (on OpenWrt pointing to the VPS) uses a domain name (rather than an IP address), the DNS server must be accessable before the WG tunnel starts.... otherwise it cannot lookup the VPS by its domain name and thus can't start the tunnel.

For some context, see this.

Is the modem a combo unit (modem+router)? Are your lan devices directly connected to the OpenWrt router or to the modem device?