I have a ddns entry that points to my ISP's modem, and the modem is set up to port forward to my openwrt router (not bridge mode), which in turn forwards http and https traffic to my NAS. The only outgoing cable from the modem is the router, so all network traffic goes through the router. When I curl the ddns address, it responds from the NAS, but when I visit that address from a browser (chrome, firefox, edge), it responds with the modem's web config ui. I tried adding a hostname entry for the private ip of the NAS to the ddns address via luci, and it again worked everywhere but the windows browsers (powershell and wsl ping resolve to my private ip, but browsers still return the modem ui). I've run ipconfig /flushdns to ensure nothing weird is left behind. I've never used edge before today, so I don't think there's any risk of lingering dns there either. I've also tried the same from my Mac and from my phone, and it works correctly on both, wired and wireless. I don't know if this is a openwrt problem or a Windows problem.
Edit: I may have had entries cached when I tested my Mac(?). It now behaves the same as the Windows machine, but the phone still responds from the NAS as expected
The modem, intelligently, does not forward http or https traffic to the NAS when it comes from the LAN, otherwise it would be impossible to configure the modem once you set up port forwarding. I don't know how it does this, but it seems relevant
Are you doing your ddns browsing testing from a device in outer internet? Like a phone without wlan. By your own words, it sounds like a test from a device inside the local network would fail just like it now does...
Ddns only helps finding the IP address for the given textual address, but ddns knows nothing about routing. Your devices handle that quite separately.
If ddns is being updated from the ISP modem, then you can try the following.
create a domain entry for the name you are using and point it to the internal lan IP of the NAS.
Make sure all the devices in the lan use OpenWrt for nameserver. If needed hijack queries to outside nameservers. Some devices like Android have GoogleDNS hardcoded and won't stick to the NS advertised by dhcp. Or some browsers are using dns over https, which will need to be disabled.
When I test from my phone, it works whether or not I'm connected to wireless or LTE. When I test from my laptop, it works when I connect when on wireless. When I test from my desktop (wired) or wired from my laptop, it responds with the modem's UI.
My DDNS is set up in openwrt, setting home.example.com to my public IP
I followed the firewall step in the link provided, and one problem I was experiencing yesterday went away (chromecast did not trust my NAS lets encrypt cert, which is what began this whole investigation, I figured that it may be using the modem UI cert), but the browser problem still exists. I did not follow the nat6 instructions because I didn't know how to undo them if it caused a problem.
Additional info that may be relevant: the modem is zyxel c3000z, and I set it up with a lan subnet for port 4 which allows the openwrt router to see the modem as existing at 192.168.1.1 instead of its default of 192.168.0.1. I'm also wondering if I misconfigured that somehow, since the problem only appears when on a wired connection (I'm still not entirely sure about this point)