Gee I've spent some time digging through the network guide and forums posts but I can't find an answer to my question. It could be because I don't actually know how to ask it! So anyway thanks for your patience, and here goes.
My cable modem is happens to be in a cupboard under the stairs, because that's the one and only place I have a coax cable jack; it then connects to the special magical yellow WAN port on my LinkSys WRT3200ACM via ethernet patch cable.
Also in this cupboard under the stairs, I have a 16-port gigabit ethernet switch that ties together a half dozen ethernet jacks I have throughout the house. This 16-port switch is connected to a normal non-yellow port on my WRT3200ACM via another ethernet patch cable.
This seems to work just fine -- both my wireless and wired internet traffic all magically flows over the OpenVpn client I run on the WRT3200ACM.
But I would like to have the WRT3200ACM in another room, where it can provide better wifi signal coverage -- yet, still supply the internet connection from the cable modem, and still transmit all internet traffic (both wired and wireless) over its OpenVpn setup.
Is this possible? It seems like not, because if I separate the cable modem from the router, I can no longer connect the modem to the router's special yellow WAN port. Unless there's an easy trick that doesn't involve snaking a 100-yard ethernet cable under a baseboard or something!
It's possible, but the details depend on the fine-print.
The easiest option would be, if you had two cables running to the same room (close enough to both reach the wrt3200acm easily). In that case you'd connect one of them 1:1 between cable modem and WAN of your wrt3200acm and the other from its LAN port back to the 16 port switch.
The second option would be adding one or more access points (connected via one of your existing cables) where it suits you and leaving the wrt3200acm where it is. Just about any OpenWrt supported device can be configured that way, but there are also commercial APs available (usually higher priced) or 'repeaters' with an ethernet port(!) which may allow being configured as AP. Technically this probably is the best option, as you significantly improve the coverage of your house- and property.
The third option would require a(n at least smart-) managed switch in place of (or in addition to) your 16-port switch, which would allow port-trunking via VLANs, basically sending two independent signals over the wire to your router (seperating WAN and LAN there, and sending the LAN data back over the same cable to the switch to distribute untagged to the rest of your house. This later option needs a slightly more advanced configuration on router- and switch and might slightly affect your maximum throughput above ~500 MBit/s WAN speed, but is otherwise a proven method (not all (smart-)managed switches can be considered safe enough to use on the WAN side (management interface bleeding over to all ports), so it needs a little more attention in this regard as well).
Ethernet cables have about a 100m range limit (you can get boosters etc but thats another story)
You can indeed patch the modems yellow Ethernet port to your router through your existing structured cabling (the jacks in the different rooms.)
Ideally you will need a dual jack in that room. One that you patch directly to the cable modem under the stairs, and the 2nd you patch into your switch. then you wire those jacks to your router as if it was under the stairs (the yellow WAN port to the modem and the plain LAN side to the switch).
The WAN Ethernet traffic should work over a pair of Homeplugs, but will probably degrade somewhat.
When running cables from one side of the house to another it may be easier to take the cable around the outside walls. Ideally put it in some trunking for mechanical and UV protection.
EDIT: You could also extend the coax of course. Coax doesn't like sharp bends, so new length around the outside of the house with a "relaxed" loop into the current terminator is what you'd be after.
The simpler option (with spending money) would be to leave it as is and use your wired ethernet to put additional dumb APs around to get better signal.
The easy trick is buying additional wireless access point devices (routers or wifi extenders, does not matter), connecting them in other rooms where you have a ethernet port and configuring them as "dumb ap" https://openwrt.org/docs/guide-user/network/wifi/dumbap aka just provide wifi to an existing network.
Since you are getting devices to do a "dumb" job where the only thing they do is generate a wifi, they don't need to be powerful or expensive like your main router that needs a strong CPU and more RAM and whatnot for doing the routing, the VPNs and everything else.
You can usually get away with 20-30 euro/dollars apiece, or less if you buy used.
You are discovering first hand why in "professional" setups the router is not the device that generates the wifi.
An important thing to keep in mind here is that you should NOT connect the modem to the switch to get to the jack in the different room, as this exposes the switch to the outside world. Now its only barrier from the Internet is the modem, which may or may not (usually may not) be very secure.
Thanks for all the great ideas! It sounds like using some dumb APs would be the best bet, followed by relocating the cable modem (and main wifi router) to a not-under-the-stairs location. The latter wouldn't be super difficult because I can access the coax in the attic and drop it down a stud bay to an upstairs location near an existing lan port.