The r7800 is right at the limits of what it can do - or probably just can't do anymore, with PPPoE entering the picture; if ~500 MBit/s is your current need (with more potential in the wires), ipq806x is beyond its abilities. Also consider if there is a way to split off routing (wired-only) and AP (on a wired/ cat-6 backhaul) into two devices, each of them placed in more adequate locations.
In terms of maximum range, don't expect that much of a difference between devices - that is limited by regulations and physics (wall penetration). Yes, 4x4 has a bit more of a margin to find the optimum connection, yes 802.11ac and 802.11ax might push a bit more throughput over the range, but you can't expect more fundamental range improvements by replacing your router (unless the old router is really on the low-end side). Wired backhauls between (multiple-) APs/ extenders (satellites) are always preferred over wireless backhauls, if you can accomodate the former, you open your selection to much cheaper (dual-radio) devices (instead of tri-radio ones, which you're not going to find among the currently supported devices with the expected routing performance).
With SQM at >500 MBit/s, you're firmly in x86_64 or highend ARMv8 territory, both of which tend not to have (usable) wireless support in the same device.
Your border gateway should not be considered a general purpose server, keep the attack surface low, don't overload it with non-routing tasks - even if it would have the performance to do that. Security and ongoing maintenance becomes a nightmare on that road.
might be worth looking at, maybe https://openwrt.org/toh/linksys/e8450 or https://openwrt.org/toh/dynalink/dl-wrx36 (don't go below ipq8072a, ipq8071a will not meet your current performance requirements). If you 'just' need APs to extend a more capable wired router, a lot more devices from cheaper sectors enter the picture.
Rockchip, RPi4 or x86_64 would be worth looking at, the options (especially the later ones, not the AMD Jaguar based ones it starts out with) from Tips for getting cheap used x86-based firewall with full Gbit NAT (a PC Engines APU) if you are in the US are always worth a look.