ER-X as core switch?

I want to build a lab with two ER-X , one as core switch , another one as internet router , here is the simple architecture

I want all traffic from 192.168.1.0 , 192.168.0.0 , 192.168.11.0 can go to internet or manage by openwrt.
eg allow 192.168.1.100-200 access internet , but 192.168.0.100-200 can not.
and all traffic between LAN are all allowed.

I use ansible to manage devices ,so I also want to use ansible to manage these two ER-X.
but I have no idea how to do so.

can anybody just give me a hint or any examples ? (not only offical documnets , please)

Is your problem with ansible-openwrt integration or with openwrt side of configuration?

If I could know how to set up openwrt correctly , then I can handle the ansible part.
so , I need instruction/samples to show me how to configure openwrt , thanks !

Are you using 2 er-x units or just one and another router such as one provided by the isp?

one ER-X as switch , and another ER-X as router

Are the ports on the router ER-X already being used? If not, there's really no need to use the second ER-X to break out the networks as you have drawn... this can all be done on the first one unless you have a reason to use the second one (i.e. more ports required, or a situation where a smart switch is needed in a physically different location than the router).

All of your networks need to be configured on the first (router) ER-X. From there, the second one will simply be a VLAN aware smart switch.

Are you running OpenWrt on both ER-X units, or do you have EdgeOS on the one of them (and if so, which one)?

Are the ports on the router ER-X already being used?

I plan to apply for three or maybe more ADSL lines , all ADSL attached to the ER-X Router, so yes, most ports are being used.

Are you running OpenWrt on both ER-X units

both ER-X running openwrt.

What I would do (no telling if this is correct or the best way…):

Use the device that connects to your internet providers as a switch only, and bridge each port / provider to a specific VLAN. Example ADSL provider 1 connected to vlan 10.
Something like (untested, and guide only):

# part of /etc/config/network for switch device
config device
	option name 'br-wantelco1'
	option type 'bridge'
	list ports 'eth1.10' # use eth1 to connect to router / second OpenWrt device
	list ports 'eth2' # telco1 provider upstream device
# you may need an interface to make the bridge come up
config interface 'wantelco1'
	option device 'br-wantelco1'
	option proto 'none'

On your router device, create a (WAN_$provider) interface for each provider. Create a device and interface for each network (1.0/24, 11.0/24, 0.0/24).

# part of /etc/config/network for router device

config interface 'wantelco1'
	option device 'eth1.10' # eth1 connects to switch device, where VLAN 10 was set for telco1
	option proto 'static'

If an interface does not have a fowarding rule in firewall config (example config forwarding net11 to wantelcoprovider), that network will not get internet access.

You will have to poke around manually to work out your config first. See https://openwrt.org/docs/guide-user/base-system/uci#configuration_files
You will need changes in all of:

  • /etc/config/network
  • /etc/config/dhcp
  • /etc/config/firewall

I would then use the openwrt uci program on device to apply the config. Otherwise you could directly modify the /etc/config/* files.

You could probably get a much cleaner config using bridge-vlans, with vlan_filtering set, but this is not yet well documented.
If you are using multiple providers, you will need to set a metric for each provider interface, or use mwan3
The mt7621 the ER-X is based on is not especially powerful, so it may limit your bandwidth when things get complex.

Have fun.

Example `uci` based shell script for config for one of my devices:
. /lib/functions.sh
. /lib/functions/system.sh

set_shed_network_bridge_ports() {
	local device="$1"
	local name ports
	config_get name "$device" name
	config_get ports "$device" ports

	if [ "$name" = "br-lan" ]; then
		if list_contains ports "sfp.6"; then
			echo "sfp with vlan set"
		else
			uci -q add_list "network.$device.ports"="sfp.6"
		fi
	fi

	if [ "$name" = "br-wan" ]; then
		if list_contains ports "sfp"; then
			uci -q del_list "network.$device.ports"="sfp"
		fi
		if list_contains ports "wan"; then
			uci -q del_list "network.$device.ports"="wan"
		fi
	fi
}

set_shed_network_ips() {
	uci -q delete network.lan.netmask="255.255.255.0"
	uci -q set network.lan.ipaddr="10.53.6.1/24"
}

set_shed_network_wannbn() {
	local interface="wannbn"
	uci -q batch <<-EOF
		set "network.br_$interface"="device"
		set "network.br_$interface.name"="br-$interface"
		set "network.br_$interface.type"="bridge"
		add_list "network.br_$interface.ports"="wan"
		add_list "network.br_$interface.ports"="sfp.9"
		set "network.$interface"="interface"
		set "network.$interface.device"="br-$interface"
		set "network.$interface.proto"="none"
	EOF

	local interface="wannbn"
	uci -q batch <<-EOF
		set "firewall.$interface"="zone"
		set "firewall.$interface.name"="$interface"
		delete "firewall.$interface.network"
		add_list "firewall.$interface.network"="$interface"
		set "firewall.$interface.input"="REJECT"
		set "firewall.$interface.output"="REJECT"
		set "firewall.$interface.forward"="REJECT"

		set "dhcp.$interface"="dhcp"
		set "dhcp.$interface.interface"="$interface"
		set "dhcp.$interface.ignore"="1"
		set "dhcp.$interface.dhcpv6"="disabled"
		set "dhcp.$interface.dhcpv4"="disabled"
		set "dhcp.$interface.ra"="disabled"
	EOF
}

set_shed_network_wifi() {
	local interface="wifi"
	local device="br_wifi"
	local device_name="br-wifi"
	uci -q batch <<-EOF
		set "network.$device"="device"
		set "network.$device".name="$device_name"
		set "network.$device".type="bridge"
		delete "network.$device".ports
		add_list "network.$device".ports="lan5.20"
		add_list "network.$device".ports="sfp.20"

		set "network.$interface"="interface"
		set "network.$interface.device"="$device_name"
		set "network.$interface.proto"="dhcp"
		set "network.$interface.metric"="2048"

		set "firewall.$interface"="zone"
		set "firewall.$interface.name"="$interface"
		delete "firewall.$interface.network"
		add_list "firewall.$interface.network"="$interface"
		set "firewall.$interface.input"="ACCEPT"
		set "firewall.$interface.output"="ACCEPT"
		set "firewall.$interface.forward"="ACCEPT"
		set "firewall.$interface.masq"="1"
		set "firewall.$interface.mtu_fix"="1"

		set "firewall.${interface}_forwarding"="forwarding"
		set "firewall.${interface}_forwarding.src"="lan"
		set "firewall.${interface}_forwarding.dest"="$interface"

		set "dhcp.$interface"="dhcp"
		set "dhcp.$interface.interface"="$interface"
		set "dhcp.$interface.ignore"="1"
		set "dhcp.$interface.dhcpv6"="disabled"
		set "dhcp.$interface.dhcpv4"="disabled"
		set "dhcp.$interface.ra"="disabled"
	EOF
}

set_shed_system() {
	uci -q batch <<-EOF
		set system.poe_passthrough.value="1"
		set system.@system[0].hostname="shed.internal.example.com"
		set system.@system[0].timezone="EST-10"
		set system.ntp.enabled="1"
		delete system.ntp.server
		add_list system.ntp.server='10.53.20.1'
		add_list system.ntp.server='0.ntp.internal.example.com'
		add_list system.ntp.server='0.au.pool.ntp.org'
	EOF
}


echo_args() {
	echo "$@"
}

config_load network
#config_foreach echo_args device

config_foreach set_shed_network_bridge_ports device
set_shed_network_ips
set_shed_network_wannbn
set_shed_network_wifi
set_shed_system

uci commit

/etc/init.d/network reload
/etc/init.d/firewall reload
/etc/init.d/gpio_switch reload

Using multiple DSL lines to get more bandwidth for users is called load balancing. You should also ask the DSL company if they offer a "bonded" service, which is better than load balancing. Bonding joins two or more copper pairs to the same modem, which operate seamlessly in parallel creating a single faster connection.

I would set up this network with one ER-X and a managed switch bringing the multiple LANs out of the ER-X on a single port tagged with VLANs.

since you want to use Ansible, you will be using command line and modifying UCI config files on the devices so I'm showing only that to do configuration.

From your picture I see that each port is called ethX and appears as a separate device. If that is the case, your device's onboard switch is using DSA and not the old "swconfig" for configuration (this will be useful if you search info about setting VLANs later, for now it does not matter)

So the first thing is splitting all ports to a separate interface so each port can have its IP address and dedicated DHCP (if you enable it)

I assume that you start with most ports bridged into a single interface called LAN, and one port is connected to an interface called WAN

First you free the other interfaces from the LAN

This is an example of the interfaces in /etc/config/network

config interface 'lan'
	option type 'bridge'
	option proto 'static'
	option netmask '255.255.0.0'
	option ipaddr '192.168.99.1'
	option ifname 'eth0'

config interface 'first'
	option type 'bridge'
	option proto 'static'
	option netmask '255.255.0.0'
	option ipaddr '192.168.0.254'
	option ifname 'eth3'

config interface 'second'
	option type 'bridge'
	option proto 'static'
	option netmask '255.255.0.0'
	option ipaddr 192.168.1.254'
	option ifname 'eth1'

config interface 'third'
	option type 'bridge'
	option proto 'static'
	option netmask '255.255.0.0'
	option ipaddr '192.168.11.254'
	option ifname 'eth2'

Then in firewall config you move all these new interfaces into the lan firewall zone. This allows them to see each other

config zone
	option name 'lan'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'
	list network 'lan'
	list network 'first'
	list network 'second'
	list network 'third'

and then you can add a rule to the firewall config to block access to WAN with the list of IPs you want to ban

config rule
	option name 'blockwan'
	option src 'lan'
	list src_ip '192.168.11.11'
	list src_ip '192.168.11.220'
	list src_ip '192.168.11.221'
	option dest 'wan'
	option target 'DROP'

And if the provider does not offer bonding you can make your own bonding with OpenMPTCP Router project and a VPS in the cloud as the other bonding node and exit point

Thank you, your reply has benefited me a lot