Towards and -especially- beyond (where it probably becomes unchallenged) 1 GBit/s WAN speeds, separating these functionalities as you laid out does make sense.
- modem
- whatever matches your ISP's- and speed requirements, as long as it can be configured to be just a modem, with the external WAN IP being terminated on your router
- router
- capable x86_64 with 2+ ethernet ports of the desired speed (so >=2.5GBASE-T in your case)
new N95/ N100 systems with 4 2.5GBASE-T ethernet ports (150-250 EUR/ USD on the big "market places") or cheap/ used SFF brandname systems (haswell i3/ i5 or newer) with slim-bracket ethernet cards in the PCIe slots come to mind.
- capable x86_64 with 2+ ethernet ports of the desired speed (so >=2.5GBASE-T in your case)
- dedicated 802.11ax wireless APs (respectively wifi routers configured as AP)
- as per your requirements (wired backhaul strongly preferred)
multiple cheaper ones in different locations tend to beat a single high-end one, as long as your can connect them to a wired backbone)
- as per your requirements (wired backhaul strongly preferred)
Managed switches widen your options when it comes to pushing multiple networks over a single trunk port, e.g. lan-, guest-, voip, surveillance cameras and IoT networks pushed through to all of your APs (VLANs ~= managed switches required). While you can implement a poor man's solution of this using multiple ethernet ports (-cards) on the router, each with their own subnet (no VLANs, plain access ports) and unmanage switches from there, managed switches make this a lot more flexible.
For 1000BASE-T and considering the prices for used (even OpenWrt capable-) L2 managed switches with 8-24 ports, I would recommend going the (L2-) managed way. For 2.5GBASE-T and beyond, prices for unmanaged switches are already quite high, with managed options excessively so - here it really depends on your budget and requirements. A hybrid approach with a large (16-24 ports) 1000BASE-T (L2-) managed switch (PoE or not) and a small (~4-8 ports) 2.5GBASE-T or 10GBASE-T 'multi-gig' switch to connect your fast desktop computers/ servers might be sensible. Under good circumstances (short range, little interference, HE80), your typical 2x2 clients (as found in notebooks/ desktops) may push 700-800 MBit/s over wifi6 (you'll only profit from wifi6e/ 6 GHz in congested environments, via reduced interference/ congestions, not top speed), so the AP backbone might still use the 1000BASE-T switch without losing much.
The rest depends on:
- your budget
- your expectations
e.g. the things you have not told us, as in sqm/cake (~QoS) desired, router-side VPN gateway, adblocking, IDS features, … - your local environment (area to cover, number of rooms/ walls, building materials, outside coverage)
- your location (densely populated apartment building --> high interference/ congestions --> 6 GHz may be beneficial, no neighbours in sight --> 5 GHz will do easily)
- regional availability and -pricing of potentially interesting devices
- your willingness to get down to the metal and set this up, as well as to maintain it long term
- abilities to install a wired ethernet backbone throughout the house
- potential WAF related conflicts of interests
Once you've figured this out, take a step back and reconsider your needs, your expectations, time- and efforts required and your budgets, you might come to the conclusion that a simpler/ flatter (1 GBit/s) topology might do 90% of what you want it to do, for 10% of the budget and 5% of the effort.
--
I've gone with an x86_64 router, wired backbone, L2 managed switches running OpenWrt and dedicated APs for a WAN speed well below the 1 GBit/s barrier myself and don't want to look back from this level of service separation again, but I still look at the infrastructure as a whole, keeping it sensible and fault tolerant (with easily replaceable components). While I'd like to push beyond 1 GBit/s lan-side, prices are not sensible for that, yet.