As already pointed out, this belief is incorrect - you can very well get modern x86_64 devices which chug under 10 watts out of the wall (down to roughly half of that) in practice.
In those hardware generations, 120-130 watts idle power consumption where indeed rather common, but the times have changed. Sandy-bridge brought that down to ~75 watts, ivy-bridge to ~30 watts, since haswell you can go down to 11-15 watts. On the Atom side, ~20 watts were possible with the venerable N270, since baytrail-d (j1900) you can achieve 6 watts (and maybe even less).
ARM is not a guarantee for low power, I have seen ARMv8 based wireless routers using around 6 watts, but others that idle around the 25 watt mark - it all depends on the hardware, the SOC, the kind of wired- and wireless capabilities.
Yes, an all-in-one 'traditional' plastic router has every opportunity to win against a system of x86_64 router, managed switch and AP(s) in terms of power consumption, but the delta doesn't need to be significant - and once you cross the border of 4+1 ethernet ports and a single AP, it might disappear altogether. Towards the upper end of the WAN speed scale, ARM based routers have a hard time to compete with x86_64 in terms of routing throughput and their ability to perform traffic shaping (sqm), x86_64 just has a lot more performance in the back (at least in regards to systems you can actually purchase as mere mortal). Tips for getting cheap used x86-based firewall with full Gbit NAT (a PC Engines APU) if you are in the US has a number of low-power examples that can compete quite well on all fronts, once you detach yourself from the dated PC engines designs in the title (e.g. sophos, gateprotect, etc.; we are talking about 6-15 watts idle depending on the actual model and four 1000BASE-T ports) - even compared against RPi4 or NanoPi r4s/ r5s setups (which also need switches and APs in addition, rendering the power consumption deltas quite negligible).