Network Bandwidth halved when routing between VLAN networks?

[kvoit]

I am using a TP-Link Archer C2600 to route between two VLANs (DMZ and LAN) that are both connected to the internal switch on separate ports. No fancy traffic rules, just general allow/reject. With iperf, I now see that bandwidth between those networks is limited to the upper 4xx MBit/s range, which is suspiciously half the expected bandwidth.
The respective networks allow around 980MBit/s, so it seems that the router is the bottleneck. I understand that the CPU is connected to the switch with a single 1GBit port, however, I would have expected that this is FullDuplex, allowing for 1GBit inbound from the DMZ and 1GBit outbound routed traffic to the LAN. Is that a misunderstanding, or should this be possible in general and I have to find the actual bottleneck?

[psherman]

Switching speeds is often faster than routing speeds.

I cannot speak to the internal architecture of the C2600, but the Ubiquiti EdgeRouter X has a 1Gb/s link between the CPU and the switch, and it can achieve only 1Gbps total routing bandwidth (not the 1Gbps+1Gb/s full duplex bandwidth that would be afforded by the forwarding performance of the switch).

What speed do you get from your ISP? And what is your routing speed for the internet?

[kvoit]

This is routing between two internal networks. There is also no NAT involved.

For the WAN interface, the device has a second hardware NIC. I am pretty sure I got full bandwidth from there before, when my DMZ was between the ISPs router and my OpenWRT device. Will test this again first thing tomorrow morning.

[psherman]

@kvoit

This is routing between two internal networks. There is also no NAT involved.

While NAT can contribute to a reduction in the routing bandwidth, it is not the only factor. That's why I'm asking about your actual ISP speeds (what you pay for and what you get when you test) to see if it is on par.

[slh]

First of all, the total routing speed of ipq8065 (without NSS offloading) is around 500-650 MBit/s (depending on the details), for everything. ipq8064, as in the c2600, is clocked 300 MHz lower, which loses you (give or take) around 100 MBit/s - which is close to your figures already.

ipq806x has two CPU ports to a single, shared, switch - in general this is assigned as one port exclusively for the WAN port and the second one exclusively to the LAN ports (but you can assign them freely between your VLANs), but this is rather immaterial considering the ceiling of ipq8064's routing throughput. Switched traffic is offloaded to the managed QCA8337 switch onboard of your c2600, this traffic can be switched at line speed (~930 MBit/s) - but routing needs to go through the SOC and its performance ceiling applies.

tl;dr: Your performance figures are within the expected values, if you need faster routing throughput, you will need (considerably) faster hardware.

[kvoit]

Thanks. As suggested, I measured WAN-to-LAN traffic now, and it completely matches your description, NAT does not further reduce it, BTW.
As I said, I had the DMZ running behind the WAN interface before, but I must have never measured the bandwidth ... Well, at least my new setup did not decrease performance ...
I always thought the SoC was reasonably powerful, because the Wireguard performance is quite solid.

All of this is preparation for a new fiber connection with which I will get rid of the ISP router in front of the C2600. After the promo period, the connection will not exceed 500MBit for now, but can someone still point me to an OpenWRT-capable router/SoC that can handle a Gigabit connection?

[kvoit]

Note: Software flow offloading gives another 70-80MBit. Hardware flow offloading doesn't seem to be implemented/present, it does not change anything.

The Xiaomi AX3200 RB01 seems reasonably cheap and some resources suggest the MT7622 supports hardware flow offloading. I would prefer something with 5+ ports, though, but I could do with VLAN trunking and probably still have better performance ...
A router-only solution like the TL-R605 without Wifi should be fine, too, as I will probably switch the wifi to Unifi at some point.

[slh]

So you have 500Mbps-1Gbps fiber and need a router READ THIS FIRST

It's such a frequently asked question over the last few years that we really need a post we can point people to. So your cable company rolled out 500Mbps or 1Gbps download speeds, or you have 1Gbps symmetric fiber (GPON/EPON) from ATT or Centurylink or Orange or whoever. You realize your old all in one router from 2009 is not up to the task of handling this, but hey by now stuff should be cheap and available that will handle your new fiber connection right? So you want advice about a router for…

Tips for getting cheap used x86-based firewall with full Gbit NAT (a PC Engines APU) if you are in the US

While I was watching for old stuff I could play with on ebay I noticed that a few sellers are offering a few "cloud managed routers" (i.e. if you don't pay a subscription they are paperweights) from a company called "simplewan" that looked oddly familiar. here some pictures (from auctions, not mine) they are bright orange. [simplewan 3] [simplewan2] [simplewan 1] The images are a dead giveaway, these devices seem to just be a branded version of PCEngines APU1 (the ones with black USB 2.0 p…

Do consider early if you may want/ need sqm, as that raises the bar even more.

[kvoit]

Thanks. It looks like I will stay with my current hardware until the actual fiber connection arrives, then possibly spend the 60€ for the Xiaomi device that probably barely makes it and otherwise wait for more powerful devices to arrive.

It's strange. 10-15 years ago, some people argued in forums that Gbit ethernet was pointless because it was faster than HDDs, and now consumer LAN hardware has trouble catching up to the freaking internet connection ... what a time to be alive ...