OK, I figured out something that can lead to the solution.
On the wifi AP, there is also a ooma device connected to one of the ethernet ports. The ooma device is a 100Mbps ethernet connection as reported by openwrt. As soon as I disconnected the Ooma, I was able to get consistently between 300-400Mbps to the internet in the wifi devices.
So, in summary, it seems like if the 100Mbps in the Ooma device was forcing the openwrt bridge to 100Mbps traffic, but only affecting traffic with IPs that were not in the same subnet (towards the Internet). Traffic within the same subnet, crossing the wifi ap into the main router to wired computers was giving a good throughput > 300Mbps.
I don't understand why such behavior.
This is the output of ip link
. I wonder if it has to do with the queuing discipline setup.
ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether da:89:ff:08:10:d2 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether 16:23:82:9d:7e:44 brd ff:ff:ff:ff:ff:ff
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP qlen 1000
link/ether 78:d2:94:0e:04:c1 brd ff:ff:ff:ff:ff:ff
5: lan1@eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master br-lan state LOWERLAYERDOWN qlen 1000
link/ether 78:d2:94:0e:04:c1 brd ff:ff:ff:ff:ff:ff
6: lan2@eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master br-lan state LOWERLAYERDOWN qlen 1000
link/ether 78:d2:94:0e:04:c1 brd ff:ff:ff:ff:ff:ff
7: lan3@eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master br-lan state LOWERLAYERDOWN qlen 1000
link/ether 78:d2:94:0e:04:c1 brd ff:ff:ff:ff:ff:ff
8: lan4@eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master br-lan state LOWERLAYERDOWN qlen 1000
link/ether 78:d2:94:0e:04:c1 brd ff:ff:ff:ff:ff:ff
9: wan@eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-lan state UP qlen 1000
link/ether 78:d2:94:0e:04:c2 brd ff:ff:ff:ff:ff:ff
10: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br-lan state UP qlen 1000
link/ether 78:d2:94:0e:04:c1 brd ff:ff:ff:ff:ff:ff
11: wlan1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br-lan state UP qlen 1000
link/ether 78:d2:94:0e:04:c0 brd ff:ff:ff:ff:ff:ff
12: wlan2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br-lan state UP qlen 1000
link/ether 78:d2:94:0e:04:bf brd ff:ff:ff:ff:ff:ff
15: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
link/ether 78:d2:94:0e:04:c1 brd ff:ff:ff:ff:ff:ff